Case Study- Interactive studies 6000 Human Resource Management

Case Study- Interactive studies 6000 Human Resource Management


M I C H A E L A . R O B E R T O

Billy Beane: Changing the Game

In 2002 Major League Baseball players and team owners negotiated a new collective bargaining agreement. During the deliberations, the owners pushed very hard for some form of enhanced revenue sharing among the 30 major league teams. They argued that a serious competitive imbalance existed on the field because large market teams such as the New York Yankees had substantially higher revenue-generating capabilities than small market teams such as the Kansas City Royals or Pittsburgh Pirates. In fact, the New York Yankees spent approximately $112 million on player salaries in 2001, while six teams spent less than $40 million.1 The owners wanted large market teams to share revenues from local radio and television contracts, as well as from other sources, with small market teams so as to diminish the gap in payrolls among teams.

Did baseball have a competitive balance problem at that time? The owners pointed to the results on the field. From 1995-2001, only four clubs with a payroll less than the major league baseball median were able to make the playoffs, and those teams managed only five wins among them in the postseason during that era.2 The Yankees, with the highest payroll in baseball, won four championships during this six-year stretch. The other two champions – the Atlanta Braves and Arizona Diamondbacks – also had payrolls near the top of the league. Meanwhile, storied franchises such as the Royals ($35 million payroll in 2001) and Pirates ($57 million payroll) had not made the playoffs since 1985 and 1992 respectively.

One team, however, seemed to defy the laws of baseball economics. The Oakland A’s had won 102 games and lost only 60 in 2001, while spending only $34 million – the 2nd lowest payroll in major league baseball! They finished first in their division and made the playoffs. The Oakland A’s spent $331,000 per win in 2001. Amazingly, the Texas Rangers, one of the highest payroll teams in baseball, spent roughly four times as much per win in that season!3 The 2001 campaign did not represent an aberration. From 2000-2003, the Oakland A’s averaged 98 wins per year, and they made it to the playoffs in each season, despite always having a payroll near the very bottom of the league.4

How did the Oakland A’s achieve such astounding success with so little money? Many people pointed to their brash general manager, Billy Beane, who had adopted a series of sophisticated statistical methodologies for evaluating players. Beane and his talented young assistants, many of whom spent more time running regression models than traveling the country scouting teenage phenoms, had sought to challenge the conventional wisdom regarding how to select players for a major league team. Specifically, they sought to identify and exploit inefficiencies in the market for baseball players, i.e., to determine what aspects of player performance were greatly overvalued and which were enormously undervalued. By doing so, the low-budget A’s found a way to compete with big-budget teams such as the New York Yankees and Boston Red Sox.

This document is authorized for use only in Integrated Studies in Human Resources Management by G. Renz from July 2009 to July 2010.

305-120 Billy Beane: Changing the Game


Billy Beane

As a teenager, Billy Beane was a highly touted young prospect that many thought would become a star in major league baseball. After graduating from high school in 1980, Beane signed a contract with the New York Mets, and he began to play in their minor league system in hopes of one day realizing his potential and becoming a member of the major league club. Beane labored under the hefty weight of lofty expectations. He no longer seemed to enjoy the game, and he began to regret his decision not to attend college.

Beane finally did get promoted to the major league team at the end of the 1984 season, but he never became a starter for the Mets. They traded him to the Minnesota Twins in 1986. For the next several years, Beane bounced back and forth between the minor leagues and the majors, for the Twins, Detroit Tigers, and Oakland A’s. In the spring of 1990, Beane finally decided to retire as a player. He requested a job as an advance scout for the A’s. As an advance scout, Beane would watch teams that Oakland would be playing in a few weeks, and he would provide his evaluations of those teams to Oakland’s manager and players, so that they could prepare for their upcoming opponents.

Beane quickly impressed people in the Oakland front office, and he became assistant general manager in 1993. Four years later, when general manager Sandy Alderson resigned to become executive vice president for Major League Baseball, Beane took his place. He had become the general manager of a major league team at age 35 — one of the youngest executives in the history of the game. As general manager, Beane was responsible for selecting the players for both the major league team as well as the franchise’s minor league clubs. He also negotiated contracts with all players, and naturally, needed to operate within a budget set by the team owners.

He had inherited a great tradition in Oakland. After all, the franchise had won three World Series championships in the 1970s and another title in 1989 during a three-year stretch in which they had played in the World Series each season. However, in the late 1980s and early 1990s, Oakland boasted one of the highest payrolls in baseball, thanks to an owner — Walter Haas, Jr. — who was quite willing to subsidize large operating losses year after year. For that reason, the team could afford such stars as Mark McGwire, Dennis Eckersley, and José Canseco. When Haas died and his estate sold the team in 1995, things changed quite dramatically. The new owners were not willing to sustain heavy financial losses for the sake of wins on the baseball diamond. The stars moved on to other teams. Oakland and its new general manager, Billy Beane, would be trying to field a competitive team despite one of the lowest payrolls in all of baseball.5 In the year prior to Beane’s hiring as general manager, the once-mighty Oakland A’s, a team now lacking in superstars, had managed to win only 65 games against 97 losses.6

Discovering Sabermetrics

Sandy Alderson served as the general manager of the Oakland A’s from 1983 to 1997. Alderson did not have the typical background of a baseball executive. He was a Harvard Law School graduate who had never played or coached in professional baseball. A former Marine infantry officer, Alderson graduated from Harvard in 1976 and practiced law in San Francisco before becoming general counsel of the A’s in 1981. Two years later, he took charge of the team. Alderson enjoyed great success in the late 1980s and early 1990s as the general manager of the A’s. In fact, the team won the World Series championship in 1989. As noted above, the franchise had one of the highest payrolls in the league, thanks to the generosity of long-time owner Walter Haas, Jr. — a man who did not mind losing money as long as the A’s kept winning.


Billy Beane: Changing the Game 305-120


During the 1980s, Alderson became very interested in the work of a relatively unknown baseball writer named Bill James. Alderson found James’ systematic approach to studying the game quite interesting and thought-provoking. He began to introduce others in the Oakland organization, including Beane, to this new stream of thinking about baseball. James had begun publishing his own writings about baseball in 1977, and he sold 75 copies of his first “book” after buying a small advertisement in The Sporting News. The “book” consisted of 68 pages that James had photocopied and stapled together himself. Through the 1980s, James began to develop a following among diehard baseball fans and a few interested sportswriters. Major league teams paid almost no attention to this writer who had never played or managed in the game.

James set out to challenge the conventional wisdom in baseball on a range of fronts. He argued that the sport had many statistics, but that most of them were quite inadequate for evaluating player performance. Fielding statistics, for instance, did not provide a good measure of a player’s ability to play defense. Baseball teams tracked the number of errors that each player made during a season. An official scorer watched each game, and that person judged whether a player failed to make a defensive play that should have been made. James wrote, “What is an error? It is, without exception, the only major statistic in sports which is a record of what an observer thinks should have been accomplished. It’s a moral judgment, really, in the peculiar quasi-morality of the locker room.”7 Michael Lewis, who wrote a book, entitled Moneyball about the Oakland A’s in 2003, described James’ argument about the inadequacy of the “error” as a statistical measure of defensive proficiency:

“A talent for avoiding obvious failure was no great trait in a big league baseball player; the easiest way not to make an error was to be too slow to reach the ball in the first place . . . The statistics were not merely inadequate; they lied. And the lies they told led the people who ran major league baseball teams to misjudge their players, and mismanage their games.”8

James set out to invent new statistics that could measure defensive proficiency. However, to accomplish this, he found that he needed to actually collect new data that did not exist. Thus, James began to track games in a completely different manner. He used the newly invented personal computer to facilitate his analysis. In 1978, James found 250 eager buyers of his second self-published book. Still, few officials in the game paid much attention at all.

Nevertheless, the concept of sabermetrics was born. Sabermetrics represented a new systematic, statistical approach to evaluating teams and players. A small cadre of baseball fanatics and hobbyists, many of them with advanced degrees in mathematics and statistics (expertise that James himself lacked), began to investigate the game in the way that James had encouraged people to do. They often wrote to James with their findings. A small, relatively tight-knit community of “baseball geeks” was born!

For instance, Dick Cramer, a research scientist in the pharmaceutical industry, began to spend his spare time crunching large datasets about baseball on his firm’s sophisticated new computers. Cramer became intrigued by the notion in baseball that some players were better “clutch hitters” than others. In other words, baseball experts and fans believed that some players performed better in crucial situations than they did in other, less critical periods of a game. Players purported to have clutch hitting ability were heralded as exceptional ballplayers. Unfortunately, Cramer’s analysis suggested that there was no such thing as clutch hitting. Few hitters were able to demonstrate statistically significant differences in their hitting prowess in “clutch” situations relative to normal playing conditions. The results were startling, yet no one in baseball accepted Cramer’s conclusions. 9 Few even became aware of them.10

James not only believed that current statistics failed to tell the “truth” about player and team performance; he also argued that “the naked eye was an inadequate tool for learning what you


305-120 Billy Beane: Changing the Game


needed to know to evaluate baseball players and baseball games.”11 For example, in baseball, a player who got a hit 30% of the time that he came to bat was considered very effective, while a player who earned a hit in only 27.5% of his appearances at the plate was considered mediocre. James explained why the naked eye could not evaluate performance effectively: “One absolutely cannot tell, by watching, the difference between a .300 hitter and a .275 hitter. The difference is one hit every two weeks. It might be that a reporter, seeing every game that the team plays, could sense that difference over the course of the year if no records were kept, but I doubt it.”12

James began to try to devise mathematical formulas that would help him predict the number of runs that a team would score, based upon various measures of hitting and base-running proficiency. His formulas revealed some startling conclusions, and they also invited others to begin employing sophisticated regression techniques to study the game. Lewis explained:

“The first version of what James called his ‘Runs Created’ formula looked like this: Runs Created = (Hits + Walks) X Total Bases/(At Bats + Walks). Crude as it was, the equation could fairly be described as a scientific hypothesis: a model that would predict the number of runs a team would score given its walks, steals, singles, doubles, etc… As it turned out, James was onto something. His model came far closer, year in and year out, to describing the run totals of every big league baseball team than anything the teams themselves had come up with. That, in turn, implied that professional baseball people had a false view of their offenses. It implied, specifically, that they didn’t place enough value on walks and extra base hits, which featured prominently in the ‘Runs Created’ model, and place too much emphasis on batting average and stolen bases, which James didn’t even bother to include . . . The details of James’ equation didn’t matter all that much. He was creating opportunities for scientists as much as doing science himself. Other, more technically adroit people, would soon generate closer approximations of reality. What mattered was (a) it was a rational, testable hypothesis; and (b) James made it so clear and interesting that it provoked a lot of intelligent people to join the conversation.”13

James not only challenged the conventional wisdom regarding how to measure player and team performance; he also demonstrated that many tried-and-true strategies employed during games were not effective. For instance, most managers chose to bunt in certain crucial situations to try to score more runs, particularly if the batter at the plate was not a proficient hitter, while the next batter was more effective. The bunt was a play in which a hitter sacrificed himself, making an out for the sake of advancing a player already on the base paths. Some teams employed the sacrifice bunt more than 100 times during a season, or more than once every two games. James’ analysis showed that the bunt, on average, did not enhance a team’s ability to score runs. In fact, it seemed to make more sense to let the batter try to earn a hit, no matter whether he was generally more or less proficient than the next hitter. While James wrote up his findings on this matter in 1979, baseball managers continued to employ the sacrifice bunt with unchanged frequency for the next two decades. By 2005, only a few teams, led by the Oakland A’s, began to de-emphasize the use of the sacrifice bunt. Most clubs continued to employ this strategy as they always had.

As Alderson began to read the work of James and his followers, he became very intrigued. However, he remained reticent about trying to apply their work to his management of the A’s. He said, “I couldn’t do a regressions analysis, but I knew what one was. And the results of them made sense to me.”14 Yet, he pointed out the problem for him: “You have to remember that there wasn’t any evidence that this shit worked. And I had credibility problems. I didn’t have a baseball background.” During the 1980s and early 1990s, Oakland’s field manager, Tony LaRussa, was considered a “genius” for his ability to win consistently from year to year. LaRussa believed fervently in many of the tenets of conventional wisdom in major league baseball.15 Moreover, the A’s had a large budget for paying players at that time. Alderson did not want to challenge LaRussa, nor


Billy Beane: Changing the Game 305-120


did he feel the need to adopt innovative and seemingly risky strategies that no one else in baseball had endorsed or accepted. However, when the team was sold in 1995, Alderson recognized that he had to adopt a different approach; otherwise the A’s would not be able compete successfully with big-budget teams.

Alderson focused first on the concept of on-base percentage. James and others had shown that batting average was not a particularly useful statistic for measuring a hitter’s proficiency, this despite the fact that Major League Baseball each year identified the players with the highest batting average in both the American and National Leagues, and they crowned those players as “batting champions.” However, batting average did not figure into James’ formulas for predicting the number of runs that a team would score.

Batting average measured the percentage of at-bats during which a player earned a hit. However, it omitted an important variable both from the numerator and denominator; namely, the number of times that a hitter earned a walk. In baseball, if the pitcher misfired on four pitches during a particular at-bat, then the hitter earned the right to go to first base – this event was called a walk. Over the years, walks were considered a statistic that measured a pitcher’s ineffectiveness; they were not viewed as an indicator of hitting prowess. James and others showed that some hitters systematically drew more walks than others. They argued that these patient hitters not only got on base more often than others, but they also became better hitters because of their patience and selectivity at the plate. Thus, James and his followers began to focus on “on-base percentage” rather than “batting average.” On-base percentage measured the frequency with which a player reached base safely, either by a hit, a walk, or some other means (such as when the pitcher struck the batter with the ball either inadvertently or intentionally).

Alderson embraced on-base percentage as a philosophy for the entire Oakland organization in the mid-1990s. Lewis explained, “Scoring runs was, in the new view, less an art or a talent than a process. If you made the process a routine – if you got every player doing his part on the production line – you could pay a lot less for runs than the going rate.”16 Alderson explained that, “The system was the star. The reason the system works is that everyone buys into it. If they don’t, there is a weakness in the system.”17 Alderson began to select players based on on-base percentage, rather than batting average, and he told his coaches at all levels to preach patience and selectivity at the plate. Soon, each minor league team in the Oakland system began to lead its league in walks, and their on-base percentage climbed higher.

Beane became a fervent believer in this new system adopted by Alderson as well as the burgeoning new science of baseball being practiced by James and his followers. He read widely on the subject, and he began to talk to some of these statistical gurus. When he became general manager of the A’s in 1997, he embraced sabermetrics as the principal tool by which he would evaluate player performance.

Beane hired several young people with mathematical and statistical expertise to help him fully implement this new approach. These young men had gone to Harvard, and they had not played or coached in professional baseball — not the usual pedigree for a baseball executive. Paul DePodesta, for instance, became a key player in the Oakland organization. DePodesta graduated from Harvard College in 1995 with a degree in economics. He played baseball and football at Harvard, and then he worked in the Canadian Football League and American Hockey League after graduation. Finally, he broke into baseball with the Cleveland Indians. Beane hired him as his assistant in 1999; DePodesta was only 26 years old, and regression analysis was his tool of choice.18


305-120 Billy Beane: Changing the Game


Challenging the Old Guard

The traditional approach to evaluating and selecting young ballplayers relied on scouting. Experienced baseball men, called scouts, traveled the country watching high school and college ballplayers in hundreds of games each year. The best scouts had been in the game for decades, and they had watched thousands of high school games in their careers. They worked feverishly to find the next hidden gem that no other scout had noticed, perhaps in some tiny rural town with a population of only a few hundred people. Experienced scouts took great pride in the players that they had discovered over the years. Of course, they often did not speak of the many “can’t-miss” young phenoms who never made it to the major leagues.

Scouts evaluated the “tools” that a player appeared to possess. A “five tool” player was considered a hot prospect. The “five tools” were: 1) hitting for average, 2) hitting for power, 3) running speed, 4) arm strength and 5) fielding ability. They used a simple checklist to indicate how many “tools” a player possessed. The scouts emphasized what they saw with their eyes, more so than the statistics that a player had compiled. After all, for high school players, statistics often proved relatively meaningless, because they depended so much on the quality of competition that a particular high school had faced. Lewis explained:

“In the scouts’ view, you found a big league ballplayer by driving sixty thousand miles, staying in a hundred crappy motels, and eating god knows how many meals at Denny’s all so you could watch 200 high school and college baseball games inside of four months, 199 of which were completely meaningless to you. Most of your worth derived from your membership in the fraternity of old scouts who did this for a living. The other little part came from the one time out of two hundred when you could walk into a ballpark, find a seat on the aluminum plank in the fourth row directly behind the catcher and see something no one else had seen — at least no one who knew the meaning of it.”19

DePodesta and Beane did not believe that scouting, with its emphasis on the use of the naked eye to evaluate the potential of very young ballplayers, was an effective way to build a club with a limited budget. Lewis explained how DePodesta’s studies at Harvard had convinced him of the limitations of scouting:

“Paul wanted to look at stats because the stats offered him new ways of understanding amateur players. He had graduated from college with distinction in economics, but his interest, discouraged by the Harvard economics department, had been on the uneasy border between psychology and economics. He was fascinated by irrationality, and the opportunities it created in human affairs for anyone who resisted it. . . There was, for starters, the tendency of everyone who actually played the game to generalize wildly from his own experience. People always thought their own experience was typical when it wasn’t. There was also a tendency to be overly influenced by a guy’s most recent performance: what he did last was not necessarily what he would do next. Thirdly — but not lastly — there was the bias toward what people saw with their own eyes, or thought they had seen. The human mind played tricks on itself when it relied exclusively on what it saw, and every trick it played was a financial opportunity for someone who saw through the illusion to the reality. There was a lot you couldn’t see when you watched a baseball game.”20

DePodesta built sophisticated statistical models that drew upon the work of writers such as Bill James. His models, like those of James, showed that on-base percentage mattered a great deal. Stolen bases did not. His analysis, though, went farther than what James had done. For instance, he invented new measures of player performance, and he was able to show the relative worth of various statistics in a way that James had not yet done. He began to demonstrate which aspects of player


Billy Beane: Changing the Game 305-120


performance were greatly overvalued in major league baseball, as well as those dimensions of performance that appeared to be undervalued. Defense and speed appeared to be quite expensive to acquire relative to their actual worth in terms of a team’s ability to score runs, prevent runs, and win games. Meanwhile, on-base percentage appeared to be relatively inexpensive to acquire, especially given its impact on a team’s ability to score runs.

DePodesta and Beane became fascinated with studying college ballplayers rather than high school prospects. College players had played many more games against better competition. They had statistics that were much more meaningful than high school players. For Beane and DePodesta, “a young player is not what he looks like, or what he might become, but what he has done.”21 They didn’t care much about a player’s body type or the speed with which they could run the 40-yard dash. Instead, they wanted to know if a player had produced in college. Could that player get on base frequently, get extra base hits, and ultimately, produce runs for his team? According to Lewis, the two men believed that they could forecast future performance of college players more effectively than high school ones:

“You could project college players with greater certainty than you could project high school players. The statistics enabled you to find your way past all sorts of sight-based scouting prejudices: the scouting dislike for short right-handed pitchers, for instance, or the scouting distrust of skinny little guys who get on base. Or the scouting distaste for fat catchers.”22

The scouts preferred high school prospects brimming with potential. They wanted to discover hidden gems with the raw skills that they felt could eventually translate into success on the baseball diamond. Dan Jennings, Vice President of Player Personnel for the Florida Marlins, described the importance of scouting and the evaluation of “tools”:

“Tools are sacred. To a scout or an organization, tools determine the value and potential impact of a prospect. An old theory of scouting has always been to ‘stick with the tools.’ It’s a simple, yet proven, theory for drafting or signing players. Scouting a ballplayer is not an exact science. Any attempt to make scouting an exact science through formulas and mathematical computations is a misguided attempt to soften the blow of the success/failure rate of the draft. . . You can’t teach tools — you can teach skills. An old track coach once said, ‘I can make anyone faster, but I can’t make anyone fast.’. . . [Skills] should never be mistaken for tools. To mistake OBP and OPS (two key statistics employed by Beane and DePodesta) for tools, is quite frankly BS.”23

Beane and DePodesta loved to hear those kinds of comments. They aspired to take advantage of the inefficiencies in the market for ballplayers. Their analysis showed that drafting young players straight out of high school was very risky. The scouts’ abilities to judge tools, and ultimately player success at the minor and major league level was spotty at best. The A’s, with their limited budget, could not afford to take many risks in the selection of ballplayers. They needed to raise the odds of signing players that would actually contribute positively at the major league level. They felt that their more objective analysis enabled them to accomplish this. It did not guarantee success, but it enhanced the probability of selecting productive ballplayers, and it insured that the team would use its resources more wisely than other teams.

The scouts in Oakland could not believe what Beane was doing. He was drafting young college players much, much higher than they felt that he should. While other teams might have a player slated to be drafted in the 15th round of the amateur draft, Beane would select that player in the first round! Moreover, Beane would approach the young man and tell him that the A’s were interested in drafting him in a high round, but only if he signed a contract for less than the going rate for someone selected at that point in the draft. Beane had incredible leverage, because no other team was


305-120 Billy Beane: Changing the Game


considering selecting that player as high in the draft. The object, then, was not to simply draft the best young ballplayers, but to select and sign to below-market contracts those prospects that were undervalued by other clubs.

Scott Hatteberg

After the 2001 season, in which the Oakland A’s had won 102 games, the team lost its star first baseman, Jason Giambi. He signed a seven year, $120 million contract with the New York Yankees. The A’s simply could not match this lucrative offer. Giambi’s contract would have eaten up approximately 40% of their entire budget for major league ballplayers! All the experts wondered how the A’s would continue to compete successfully given the loss of their superstar, who had won the American League’s Most Valuable Player Award in 2000 and had been the runner-up in 2001. Many people predicted that the A’s would not win 90 games and would not make the playoffs. Instead, Oakland won 103 games in 2002, without Giambi, and they won their division and made the playoffs again!

One key acquisition during the 2001-2002 off-season was Scott Hatteberg. He would play first base in place of Jason Giambi. Hatteberg was a back-up catcher who had played six years with the Boston Red Sox. He had gotten hurt, and he could no longer throw the ball effectively. Thus, his career as a catcher was over. The Red Sox did not attempt to re-sign him after the 2001 season. After all, Hatteberg was basically a .270 hitter with modest power — the perfect picture of mediocrity to most experts in the game.24 If he could not catch, then he was not a very desirable commodity.

Beane and DePodesta, however, had been watching Hatteberg for some time. While Hatteberg’s batting average and power statistics were modest at best, he had an uncanny knack for getting on base. He was an incredibly patient hitter who earned walks at an incredible rate. Moreover, he wore down opposing pitchers because he made them throw many pitches each time he stepped to the plate. He also did not strike out very often; he always seemed to put the bat on the ball.

Interestingly, many Red Sox managers and coaches ostracized Hatteberg for his patience at the plate. They wanted him to swing the bat more often. At one point, the hitting coach criticized Hatteberg publicly and vociferously for not swinging at the first pitch more often, given that he tended to get a hit 50% of the time that he swung at the pitcher’s first offering. Hatteberg explained his frustration with these criticisms: “He didn’t understand that the reason I hit .500 when I swung at the first pitch was that I only swung at first pitches that were too good not to swing at.”25

Oakland had a different view. The A’s offered Hatteberg a contract with a base salary of $950,000 — far below the major league average. No one else wanted him, and certainly they did not offer him a comparable salary. In 2002, in his first season with Oakland, Hatteberg had an on-base percentage of .374 — tied for 13th in the American League.26 He had gotten on base more frequently than superstars such as Derek Jeter and Nomar Garciaparra — players who earned salaries well in excess of $10 million per season!27 Only two players in the American League made pitchers throw more pitches than Hatteberg did per plate appearance. His ratio of walks to strikeouts was fourth best in the league. Yes, he only hit 15 home runs, far below the number that Giambi had hit in his final season in Oakland.28 However, Hatteberg represented one piece to the puzzle that Beane and DePodesta assembled each year. Their objective was to replace the offensive production that Giambi had contributed to the team, but not necessarily with one player. They knew only that they had to find a way to replace that run production for the team as a whole. Hatteberg represented one cog in the machine, and he was an incredibly under-valued cog at that.


Billy Beane: Changing the Game 305-120


Lavish Imitators?

With all of Oakland’s success, one might think that many teams would imitate their formula for success. However, few teams adopted Oakland’s radical approach to selecting ballplayers. They continued to rely heavily on scouting, while not embracing many of the most advanced statistical techniques devised by James, DePodesta, and others. Teams continued to employ the sacrifice bunt rather frequently, something Beane had instructed his field manager not to do.

Only three teams would move dramatically in the direction of the Oakland A’s. Two of those teams — the Los Angeles Dodgers and the Toronto Blue Jays — did so by hiring Beane’s protégés. The Blue Jays hired J.P. Riccardi from the A’s in November 2001. Riccardi promptly hired a 28-year old Harvard graduate who had never played baseball as his assistant. He also fired 25 of the team’s scouts. Riccardi would have to rebuild the Blue Jays with a limited budget, fairly similar to the payroll of the A’s.

The Dodgers hired DePodesta before the 2004 season. He was only 31 years old, four years younger than Beane had been when he became general manager of the A’s. Most general managers in major league baseball could have been DePodesta’s father or even grandfather. DePodesta, though, had one advantage over Riccardi. The Dodgers had one of the highest payrolls in baseball. However, the new owners of the team had spent a great deal on the team, and they were searching for ways to compete successfully while spending less on players. The Dodgers had spent a great deal of money throughout the past decade, with little success to show for their profligate spending. The team had not won a playoff game in 15 years.29

The third team to emulate the Oakland A’s was the Boston Red Sox. They tried to hire Beane, but after first accepting the offer, he chose to stay in Oakland for family reasons. Thus, the Red Sox promoted 28 year old Theo Epstein to the position — he became the youngest general manager in baseball history. Epstein had a history degree from Yale and had graduated from the University of San Diego Law School. He had never played professional baseball. The Red Sox were one of the highest revenue-generating teams in baseball, yet their new owner, John Henry, believed deeply in sabermetrics. He knew that Epstein too had embraced this new philosophy regarding how to evaluate and select ballplayers. As a fourth grader at an elementary school in Brookline, Massachusetts — just a short subway ride from Fenway Park — Epstein had discovered and become fascinated with the writings of Bill James.30

Henry had made his fortune as an investor, but he was also a long-time avid fan of James’ writings about baseball. Specifically, Henry had made his mark by developing sophisticated statistical techniques to identify and capitalize on inefficiencies in commodity markets. Henry explained how he saw similarities between the financial markets and major league baseball:

“People in both fields operate with beliefs and biases. To the extent that you can eliminate both and replace them with data, you gain a clear advantage. Many people think they are smarter than others in the stock market and that the market itself has no intrinsic intelligence — as if it’s inert. Many people they think they are smarter than others in baseball and that the game on the field is simply what they think it is through their set of images/beliefs. Actual data from the market means more than individual perception/belief. The same is true in baseball.”31

Henry not only hired Epstein as his general manager, but he also brought on board Bill James as a consultant to the team. Twenty-five years after he had self-published his first writings about baseball, James had finally found his way into the mainstream. He had his first job with a major


305-120 Billy Beane: Changing the Game


league club. Henry exclaimed, “I don’t understand how it took this long for somebody to hire this guy.”32 Epstein noted:

“What Bill offers us, more than a particular set of sophisticated statistical formulas, is a way of thinking. Bill doesn’t start with an assumption and then find data to support it, like a lot of people in baseball do. Bill starts with a question, and then he does the research objectively and doggedly, and let’s the truth empirically come to him.”33

Many people ridiculed Henry’s moves though. How could he trust such a young man with one of the most storied franchises in all of baseball? Was he really going to let James shape the decisions that were made by the club? It was one thing to enjoy reading this hobbyist’s idiosyncratic thoughts about baseball; it was quite another to employ his ideas to build a major league team. A New Yorker magazine article about James’ hiring described the reaction of Boston’s preeminent baseball writer: “Dan Shaughnessy, the dean of Boston Globe sportswriters, told me that he’s “dubious” of the James experiment, and that he’d even heard grumbling among the press corps about the possibility of lineups being faxed in daily from Kansas.” (Bill James lived in Kansas, of course). 34

In 2003, the Red Sox had decided not to sign a top notch closer — a relief pitcher that every team used specifically to try to finish games in which they were leading. James believed that closers were over-valued in baseball. Moreover, he was convinced that teams should employ their best relief pitcher at crucial times in any of the later innings, rather than saving that pitcher for the final inning as every team did at the time. The Sox experiment was considered a failure in 2003. The players did not like the concept. The field manager, Grady Little, refused to endorse the concept. Moreover, the lack of a proven closer definitely hurt the team in the win column. The Red Sox failed to hold leads in the ninth inning in an unusually large number of games. In the final game of the American League Championship Series, Little left his starting pitcher in for an unusually long period of time, despite clear signs that he was tiring and faltering, because he did not have confidence in his relief pitchers and did not have a proven closer upon which he could rely.35

In the off-season, Epstein signed one of the best closers in baseball to a lucrative contract. Interestingly, he signed that player away from the Oakland A’s, a team that felt, like James, that prominent closers were overvalued in major league baseball. The pitcher, Keith Foulke, went on to have a great year for the Red Sox. Moreover, Red Sox field manager Terry Francona deployed Foulke in a manner quite similar to all other managers in baseball. In short, he brought Foulke in at the end of the game, not as James recommended that teams should employ their best relief pitcher (i.e. in crucial situations even in the earlier innings).36

Of course, few people laughed at the concept of sabermetrics, or at the decision to hire Epstein and James, when the Boston Red Sox won the World Series for the first time in 86 years in October 2004, in only the second year of Epstein’s tenure as general manager. Of course, the traditionalists pointed to the fact that the Red Sox were far, far different than the Oakland A’s. They had the second-highest payroll in major league baseball, behind only the New York Yankees.37 Moreover, Foulke was on the pitcher’s mound when the final game ended, and the Red Sox clinched the championship. According to some, they had won despite Bill James, not because of him.

Supporters of sabermetrics pointed to Epstein’s signings of players such as David Ortiz and Bill Mueller. Neither player was considered a star when Epstein signed them to contracts in his first year as general manager. In fact, neither man was a full-time starter for his previous team. Ortiz went on to finish in the top 10 in American League Most Valuable Player voting in his first two seasons in Boston, and Mueller won a batting championship. Epstein had signed both players to contracts that paid far less than what superstars with similar statistics earned on other teams. Many people considered Ortiz one of the best values in baseball.38


Billy Beane: Changing the Game 305-120


The Traditionalists React

When Lewis published his book about the Oakland A’s in 2003, the reaction from most general managers, journalists, and television analysts was very negative. In Lewis’ words, they “flipped out.”39 The experts directed their wrath at Beane, rather than at Lewis. One journalist wrote that Beane “has done a terrific job with modest funds with the A’s, but he’s also a shameless self-promoter who wrote a book about his imagined genius and is despised by scouts around baseball.”40 Of course, Beane had not written the book; Lewis had!

General managers and scouts around the game scoffed at the notion of Beane as some sort of genius. One personnel executive said, “You can talk all you want about this newfangled OPS bullshit (OPS was a key hitting statistic used by sabermetricians), but I just sit there and laugh.”41 He and others did not think Oakland’s success had much to do with finding hitters that other teams did not want. They pointed to the fact that Beane had been “lucky” to draft three young prospects who became incredibly good starting pitchers. Of course, Beane felt that he had spotted their potential where other teams had overlooked them.

Opposing general managers and scouts also pointed to the fact that the A’s had won many regular season games, but had not performed well in the playoffs during Beane’s tenure. As one executive said, “They don’t have any [championship] rings, do they? They’ve got three horses in that [pitching] rotation and they’re riding the hell out of them, but they still get their butts beat every year in the first round.”42 It was true that Oakland had not won a playoff series during Beane’s tenure as general manager. Beane argued that his approach to baseball worked over a long regular season (162 games), but that success in the postseason (in a 7 game playoff series) often depended much more on chance. Traditionalists scoffed at Beane’s defense of his team’s lack of postseason success.

In the afterword to the paperback edition of his book, Lewis tried to explain the torrent of negative reaction stimulated by Moneyball:

“Baseball has structured itself less as a business than as a social club . . The Club is selective, but the criteria for admission and retention are nebulous. There are many ways to embarrass the Club, but being bad at your job isn’t one of them. The greatest offense a Club member can commit is not ineptitude but disloyalty . . . That’s not to say that there are not good baseball executives and bad baseball executives, or good baseball scouts and bad baseball scouts. It’s just that they aren’t very well sorted out.”43

Sustaining Success

As the 2005 season approached its midpoint, many questions were being raised about the sustainability of Oakland’s success. The team had failed to make the playoffs for the first time in five years during the 2004 season. Due to budget constraints, Beane had to trade two of his three star pitchers — Mark Mulder and Tim Hudson — prior to the start of the 2005 campaign. He had decided to go with younger, much less expensive pitchers to try to remain within his budget, recognizing that it would take a while for these pitching prospects to develop.44 Through 70 games of the 2005 season, the drop-off in performance proved to be substantial. Oakland stood in last place in its division, with a record of 31 wins and 39 losses.45

Oakland also faced other issues that threatened the sustainability of its operating model. In the 2002 collective bargaining agreement, the players and owners had agreed to some limited revenue sharing. According to that union contract, the large market teams had to share some of their locally generated revenue with small market teams. The owners hoped that such revenue sharing would


305-120 Billy Beane: Changing the Game


enable small market teams to raise their payrolls, and that the disparity in spending between large market and small market teams would begin to shrink. By the 2005 season, the impact of the new revenue sharing plan had grown to be quite substantial. While teams such as the New York Yankees and Boston Red Sox contributed large sums to the revenue-sharing pool (the Yankees paid $63 million into the pool at the end of the 2004 campaign), small market teams such as Milwaukee and Cincinnati received additional money to spend on payroll. Consequently, five teams increased their payrolls by 33% or more in 2005, and each of those teams received money from the revenue-sharing pool (rather than contributing as the Yankees did). 46

Oakland also faced the dismantling of the management team that had led the organization over the past decade. While Beane remained, many of his colleagues had moved on to other organizations. DePodesta and Riccardi had taken general manager positions with other teams. David Forst, another Harvard graduate and statistical guru who currently served as the assistant general manager, might very well be the next to leave the organization for greener pastures.

James, of course, had been hired by the Red Sox. That meant that some of his groundbreaking analysis would no longer sit in the public domain, available for Oakland to employ. Instead, the Red Sox would have proprietary access to James’ analysis. At the same time, many other sabermetricians were sharing their analysis widely via the Internet. Any team that was interested in the analysis could simply read it for free on the web. The question became: Could Oakland continue to develop proprietary statistics and regression models that would help them select players, or would most of the critical variables and statistical models become widely available to all interested parties?

Finally, a rise in the use of sabermetrics by other clubs might very well reduce the inefficiencies in the market for baseball players. If Oakland’s success spawned more and more imitators, then it would become very difficult to find undervalued ballplayers using the techniques adopted and pioneered by Beane and his associates. In fact, in April 2005, Sandy Alderson announced that he was stepping down after eight years as an executive in the league office and becoming the president of the San Diego Padres. Undoubtedly, sabermetrics would soon be a much more important factor in that organization’s player selection process.47

Fortunately for Oakland, Beane could take some solace in the fact that most teams seemed to continue to rely heavily on traditional scouting. Bill Stoneman, the general manager of the Anaheim Angels, explained his team’s philosophy: “What we rely on heavily are our own judgments and the judgments of our scouting people. We’re a scouting organization, and we really lean on our pro scouts as to what they see in a player. It’s really not what the guy did last year; it’s what you think he’s going to do this year or in the future.”48 Jim Beattie, executive vice president for the Baltimore Orioles, felt that some teams had become too reliant on statistics: “If anybody relies on a statistic to make their evaluation, they’re probably going to end up failing more often than not.”49

Arizona Diamondbacks general manager Joe Garagiola, Jr. summed up the feelings of many of the executives who ran major league teams:

“Maybe I’m hopelessly old school in this regard, but to me statistics that you can derive from sort of the basic building blocks I think have real value. I look at ‘Baseball Prospectus’ [a baseball magazine with a statistical focus] from time to time, and some of those stats are so arcane, dense, impenetrable — whatever the word you want to use is. I guess this is meaningful to somebody, but not me. It drills down so deeply, it’s like, ‘OK, when you hit the bottom, there are three people in the world that this matters to.’”50

Somewhere in Oakland, you just knew that Billy Beane was chuckling at Garagiola’s comments. To him, it just seemed that so many of his colleagues had their heads hopelessly buried in the sand.


Billy Beane: Changing the Game 305-120


End Notes

1 2 Alexander, Andrew. “Fixing Baseball’s Competitive Balance Problem.” The Intellectual Conservative. August

11, 2003. 3 Lewis, Michael. (2004). Moneyball: The Art of Winning An Unfair Game. New York: W.W. Norton and

Company. 4 5 Lewis, (2004). 6 7 Lewis, (2004), p. 66. 8 Lewis, (2004), p. 67. 9 Lewis, (2004). 10 The sabermetric analysis of clutch hitting did not gain widespread attention until Sports Illustrated

published a story about it in March 2004. See T. Verducci, “Does Clutch Hitting Really Exist?” Sports Illustrated, March 30, 2004. Interestingly, with attention finally on this topic, some are now questioning Cramer’s conclusions. For instance, a University of Pennsylvania student conducted a study that shows that some players are, in fact, clutch hitters. See “Clutch Hitters and Choke Hitters: Myth or Reality,” University of Pennsylvania Press Release, May 5, 2005. Moreover, Bill James has written an article that calls for a re-examination of the statistical methodology that Cramer employed, and in fact, that James has employed frequently in the past. While James does not directly challenge Cramer’s conclusions, he does expose weaknesses in the statistical approach. See James, Bill. (2004). “Underestimating the Fog,” The Baseball Research Journal, Volume 33.

11 Lewis, (2004), p. 68. 12 Lewis, (2004). p. 68. 13 Lewis, (2004). p. 77-78. 14 Lewis, (2004). p. 57. 15 For more on LaRussa’s approach to managing a game, see Bissinger, Buzz. (2005). Three Nights in August.

Boston: Houghton-Mifflin. 16 Lewis, (2004). p. 59. 17 Lewis, (2004). p. 59. 18 19 Lewis, (2004). p. 37. 20 Lewis, (2004). p. 18. 21 Lewis, (2004). p. 38. 22 Lewis, (2004). p. 38. 23 Jennings, Dan. “Jennings Relies on Tools To Shape Marlins Roster,” Baseball America, March 21, 2003. 24 25 Lewis, (2004). p. 178. 26 Lewis, (2004). p. 178.


305-120 Billy Beane: Changing the Game



28 Lewis, (2004). p. 171. 29 Painter, Jill. “Will DePodesta Bring ‘Moneyball’ to La-La Land,” Baseball America, February 17, 2004. 30 Shaughnessy, Dan. (2005). Reversing the Curse. Boston: Houghton-Mifflin. 31 Lewis, (2004). p. 90-91. 32 Neyer, Rob. “Red Sox Hire James In Advisory Capacity,”, November 5, 2002. 33 Neyer, Rob. “Red Sox Hire James In Advisory Capacity,”, November 5, 2002. 34 McGrath, Ben. “The Professor of Baseball,” The New Yorker, July 14, 2003. 35 Shaughnessy, (2005). 36 Shaughnessy, (2005). 37 In fact, the 2004 Boston Red Sox became the first World Series champion with a player payroll in excess of

$100 million. 38 Shaughnessy, (2005). 39 Lewis, (2004). p. 290. 40 Lewis, (2004). p. 290. 41 Berardino, Mike. “The Great Debate: While Sabermetrics Have Made Great Inroads In The Game, Some

Still View Statistical Analysis With Skepticism,” Baseball America, March 21, 2003. 42 Berardino, (2003). 43 Lewis, (2004). p. 287-288. 44 45 46 Bodley, Hal. “Revenue Sharing Paying Off,” USA Today, April 7, 2005. 47 Krasovic, Tom. “Padres Lure Alderson,” San Diego Union-Tribune. April 19, 2005. 48 Berardino, (2003). 49 Berardino, (2003). 50 Berardino, (2003).


<< /ASCII85EncodePages false /AllowTransparency false /AutoPositionEPSFiles true /AutoRotatePages /All /Binding /Left /CalGrayProfile (None) /CalRGBProfile (sRGB IEC61966-2.1) /CalCMYKProfile (U.S. Web Coated \050SWOP\051 v2) /sRGBProfile (sRGB IEC61966-2.1) /CannotEmbedFontPolicy /Warning /CompatibilityLevel 1.2 /CompressObjects /Tags /CompressPages true /ConvertImagesToIndexed true /PassThroughJPEGImages true /CreateJDFFile false /CreateJobTicket false /DefaultRenderingIntent /AbsoluteColorimetric /DetectBlends true /ColorConversionStrategy /LeaveColorUnchanged /DoThumbnails true /EmbedAllFonts true /EmbedJobOptions true /DSCReportingLevel 0 /EmitDSCWarnings true /EndPage -1 /ImageMemory 1048576 /LockDistillerParams false /MaxSubsetPct 100 /Optimize true /OPM 1 /ParseDSCComments true /ParseDSCCommentsForDocInfo true /PreserveCopyPage true /PreserveEPSInfo true /PreserveHalftoneInfo true /PreserveOPIComments true /PreserveOverprintSettings true /StartPage 1 /SubsetFonts false /TransferFunctionInfo /Preserve /UCRandBGInfo /Preserve /UsePrologue false /ColorSettingsFile () /AlwaysEmbed [ true /AGaramond-Bold /AGaramond-BoldItalic /AGaramond-Italic /AGaramond-Regular /AlleycatICG /AlleycatICG-Bold /Anna /AvantGarde-Book /AvantGarde-BookOblique /AvantGarde-Demi /AvantGarde-DemiOblique /BaskervilleBE-Italic /BaskervilleBE-Medium /BaskervilleBE-MediumItalic /BaskervilleBE-Regular /BernhardModern-Roman /BocaRatonICG /BocaRatonICG-Solid /BookAntiqua /BookAntiqua-Bold /BookAntiqua-BoldItalic /BookAntiqua-Italic /Bookman-Demi /Bookman-DemiItalic /Bookman-Light /Bookman-LightItalic /Boton-Italic /Boton-Medium /Boton-MediumItalic /Boton-Regular /Boulevard /CaflischScript-Bold /CaflischScript-Regular /ChiladaICG-Cuatro /ChiladaICG-Dos /ChiladaICG-Tres /ChiladaICG-Uno /Copperplate-ThirtyOneAB /Copperplate-ThirtyThreeBC /Courier /Courier-Bold /Courier-BoldOblique /Courier-Oblique /Critter /DecoturaICG /DecoturaICG-Inline /ExPonto-Regular /FajitaICG-Mild /FajitaICG-Picante /FranklinGothic-Condensed /FranklinGothic-Roman /Frutiger-Bold /Frutiger-Italic /Frutiger-Light /Frutiger-Roman /Giddyup /Giddyup-Thangs /Goudy /Helvetica /Helvetica-Bold /Helvetica-BoldOblique /Helvetica-Narrow /Helvetica-Narrow-Bold /Helvetica-Narrow-BoldOblique /Helvetica-Narrow-Oblique /Helvetica-Oblique /JansonText-Italic /JansonText-Roman /Lithos-Black /Lithos-Regular /LitterboxICG /Minion-BoldCondensed /Minion-BoldCondensedItalic /Minion-Condensed /Minion-CondensedItalic /Minion-Ornaments /Myriad-Bold /Myriad-BoldItalic /Myriad-Italic /Myriad-Roman /Myriad-Tilt /NewCenturySchlbk-Bold /NewCenturySchlbk-BoldItalic /NewCenturySchlbk-Italic /NewCenturySchlbk-Roman /Nueva-BoldExtended /Nueva-Roman /NuptialScript /OCRA /PaisleyICG-01 /PaisleyICG-01Alt /PaisleyICG-02 /PaisleyICG-02Alt /Palatino-Bold /Palatino-BoldItalic /Palatino-Italic /Palatino-Roman /ParkAvenue /Poetica-ChanceryI /PopplLaudatio-Italic /PopplLaudatio-Medium /PopplLaudatio-MediumItalic /PopplLaudatio-Regular /PrestigeElite-Bold /PrestigeElite-BoldSlanted /Sanvito-Light /Sanvito-Roman /SaturdaySansICG /SaturdaySansICG-Bold /Stencil /Tekton-Bold /TektonMM /TektonMM-Oblique /Times-Bold /Times-BoldItalic /Times-Italic /Times-Roman /Trajan-Bold /UltraCondensedSansOne /UltraCondensedSansTwo /Utopia-Italic /Utopia-Regular /Utopia-Semibold /Utopia-SemiboldItalic /VAGRounded-Black /VAGRounded-Bold /VAGRounded-Light /VAGRounded-Thin /Viva-BoldExtraExtended /Viva-Regular /WhimsyICG /WhimsyICG-Bold /WhimsyICG-Heavy /Willow /WontonICG /ZapfChancery-MediumItalic ] /NeverEmbed [ true ] /AntiAliasColorImages false /DownsampleColorImages false /ColorImageDownsampleType /Bicubic /ColorImageResolution 600 /ColorImageDepth -1 /ColorImageDownsampleThreshold 1.50000 /EncodeColorImages true /ColorImageFilter /DCTEncode /AutoFilterColorImages true /ColorImageAutoFilterStrategy /JPEG /ColorACSImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /ColorImageDict << /QFactor 2.40 /HSamples [2 1 1 2] /VSamples [2 1 1 2] >> /JPEG2000ColorACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 15 >> /JPEG2000ColorImageDict << /TileWidth 256 /TileHeight 256 /Quality 15 >> /AntiAliasGrayImages false /DownsampleGrayImages false /GrayImageDownsampleType /Bicubic /GrayImageResolution 300 /GrayImageDepth -1 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages true /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /GrayImageDict << /QFactor 2.40 /HSamples [2 1 1 2] /VSamples [2 1 1 2] >> /JPEG2000GrayACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 15 >> /JPEG2000GrayImageDict << /TileWidth 256 /TileHeight 256 /Quality 15 >> /AntiAliasMonoImages false /DownsampleMonoImages false /MonoImageDownsampleType /Bicubic /MonoImageResolution 300 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict << /K -1 >> /AllowPSXObjects true /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile (None) /PDFXOutputCondition () /PDFXRegistryName ( /PDFXTrapped /False /Description << /JPN <FEFF3053306e8a2d5b9a306f300130d330b830cd30b9658766f8306e8868793a304a3088307353705237306b90693057305f00200050004400460020658766f830924f5c62103059308b3068304d306b4f7f75283057307e305930023053306e8a2d5b9a30674f5c62103057305f00200050004400460020658766f8306f0020004100630072006f0062006100740020304a30883073002000520065006100640065007200200035002e003000204ee5964d30678868793a3067304d307e30593002> /DEU <FEFF00560065007200770065006e00640065006e0020005300690065002000640069006500730065002000450069006e007300740065006c006c0075006e00670065006e0020007a0075006d002000450072007300740065006c006c0065006e00200076006f006e0020005000440046002d0044006f006b0075006d0065006e00740065006e002c00200075006d002000650069006e00650020007a0075007600650072006c00e40073007300690067006500200041006e007a006500690067006500200075006e00640020004100750073006700610062006500200076006f006e00200047006500730063006800e40066007400730064006f006b0075006d0065006e00740065006e0020007a0075002000650072007a00690065006c0065006e002e00200044006900650020005000440046002d0044006f006b0075006d0065006e007400650020006b00f6006e006e0065006e0020006d006900740020004100630072006f0062006100740020006f0064006500720020006d00690074002000640065006d002000520065006100640065007200200035002e003000200075006e00640020006800f600680065007200200067006500f600660066006e00650074002000770065007200640065006e002e> /FRA <FEFF004f007000740069006f006e00730020007000650072006d0065007400740061006e007400200064006500200063007200e900650072002000640065007300200064006f00630075006d0065006e007400730020005000440046002000700072006f00660065007300730069006f006e006e0065006c007300200066006900610062006c0065007300200070006f007500720020006c0061002000760069007300750061006c00690073006100740069006f006e0020006500740020006c00270069006d007000720065007300730069006f006e002e00200049006c002000650073007400200070006f0073007300690062006c0065002000640027006f00750076007200690072002000630065007300200064006f00630075006d0065006e007400730020005000440046002000640061006e00730020004100630072006f0062006100740020006500740020005200650061006400650072002c002000760065007200730069006f006e002000200035002e00300020006f007500200075006c007400e9007200690065007500720065002e> /PTB <FEFF005500740069006c0069007a006500200065007300740061007300200063006f006e00660069006700750072006100e700f5006500730020007000610072006100200063007200690061007200200064006f00630075006d0065006e0074006f0073002000500044004600200063006f006d00200075006d0061002000760069007300750061006c0069007a006100e700e3006f0020006500200069006d0070007200650073007300e3006f00200061006400650071007500610064006100730020007000610072006100200064006f00630075006d0065006e0074006f007300200063006f006d0065007200630069006100690073002e0020004f007300200064006f00630075006d0065006e0074006f0073002000500044004600200070006f00640065006d0020007300650072002000610062006500720074006f007300200063006f006d0020006f0020004100630072006f006200610074002c002000520065006100640065007200200035002e00300020006500200070006f00730074006500720069006f0072002e> /DAN <FEFF004200720075006700200064006900730073006500200069006e0064007300740069006c006c0069006e006700650072002000740069006c0020006100740020006f0070007200650074007400650020005000440046002d0064006f006b0075006d0065006e007400650072002c0020006400650072002000650072002000650067006e006500640065002000740069006c0020007000e5006c006900640065006c006900670020007600690073006e0069006e00670020006f00670020007500640073006b007200690076006e0069006e006700200061006600200066006f0072007200650074006e0069006e006700730064006f006b0075006d0065006e007400650072002e0020005000440046002d0064006f006b0075006d0065006e007400650072006e00650020006b0061006e002000e50062006e006500730020006d006500640020004100630072006f0062006100740020006f0067002000520065006100640065007200200035002e00300020006f00670020006e0079006500720065002e> /NLD <FEFF004700650062007200750069006b002000640065007a006500200069006e007300740065006c006c0069006e00670065006e0020006f006d0020005000440046002d0064006f00630075006d0065006e00740065006e0020007400650020006d0061006b0065006e00200064006900650020006700650073006300680069006b00740020007a0069006a006e0020006f006d0020007a0061006b0065006c0069006a006b006500200064006f00630075006d0065006e00740065006e00200062006500740072006f0075007700620061006100720020007700650065007200200074006500200067006500760065006e00200065006e0020006100660020007400650020006400720075006b006b0065006e002e0020004400650020005000440046002d0064006f00630075006d0065006e00740065006e0020006b0075006e006e0065006e00200077006f007200640065006e002000670065006f00700065006e00640020006d006500740020004100630072006f00620061007400200065006e002000520065006100640065007200200035002e003000200065006e00200068006f006700650072002e> /ESP <FEFF0055007300650020006500730074006100730020006f007000630069006f006e006500730020007000610072006100200063007200650061007200200064006f00630075006d0065006e0074006f0073002000500044004600200071007500650020007000650072006d006900740061006e002000760069007300750061006c0069007a006100720020006500200069006d007000720069006d0069007200200063006f007200720065006300740061006d0065006e0074006500200064006f00630075006d0065006e0074006f007300200065006d00700072006500730061007200690061006c00650073002e0020004c006f007300200064006f00630075006d0065006e0074006f00730020005000440046002000730065002000700075006500640065006e00200061006200720069007200200063006f006e0020004100630072006f00620061007400200079002000520065006100640065007200200035002e003000200079002000760065007200730069006f006e0065007300200070006f00730074006500720069006f007200650073002e> /SUO <FEFF004e00e4006900640065006e002000610073006500740075007300740065006e0020006100760075006c006c006100200076006f006900740020006c0075006f006400610020006a0061002000740075006c006f00730074006100610020005000440046002d0061007300690061006b00690072006a006f006a0061002c0020006a006f006900640065006e0020006500730069006b0061007400730065006c00750020006e00e400790074007400e400e40020006c0075006f00740065007400740061007600610073007400690020006c006f00700070007500740075006c006f006b00730065006e002e0020005000440046002d0061007300690061006b00690072006a0061007400200076006f0069006400610061006e0020006100760061007400610020004100630072006f006200610074002d0020006a0061002000520065006100640065007200200035002e00300020002d006f0068006a0065006c006d0061006c006c0061002000740061006900200075007500640065006d006d0061006c006c0061002000760065007200730069006f006c006c0061002e> /ITA <FEFF00550073006100720065002000710075006500730074006500200069006d0070006f007300740061007a0069006f006e00690020007000650072002000630072006500610072006500200064006f00630075006d0065006e007400690020005000440046002000610064006100740074006900200070006500720020006c00610020007300740061006d00700061002000650020006c0061002000760069007300750061006c0069007a007a0061007a0069006f006e006500200064006900200064006f00630075006d0065006e0074006900200061007a00690065006e00640061006c0069002e0020004900200064006f00630075006d0065006e00740069002000500044004600200070006f00730073006f006e006f0020006500730073006500720065002000610070006500720074006900200063006f006e0020004100630072006f00620061007400200065002000520065006100640065007200200035002e003000200065002000760065007200730069006f006e006900200073007500630063006500730073006900760065002e> /NOR <FEFF004200720075006b00200064006900730073006500200069006e006e007300740069006c006c0069006e00670065006e0065002000740069006c002000e50020006f00700070007200650074007400650020005000440046002d0064006f006b0075006d0065006e00740065007200200073006f006d002000700061007300730065007200200066006f00720020007000e5006c006900740065006c006900670020007600690073006e0069006e00670020006f00670020007500740073006b007200690066007400200061007600200066006f0072007200650074006e0069006e006700730064006f006b0075006d0065006e007400650072002e0020005000440046002d0064006f006b0075006d0065006e00740065006e00650020006b0061006e002000e50070006e006500730020006d006500640020004100630072006f0062006100740020006f0067002000520065006100640065007200200035002e00300020006f0067002000730065006e006500720065002e> /SVE <FEFF0041006e007600e4006e00640020006400650020006800e4007200200069006e0073007400e4006c006c006e0069006e006700610072006e00610020006e00e40072002000640075002000760069006c006c00200073006b0061007000610020005000440046002d0064006f006b0075006d0065006e007400200073006f006d00200070006100730073006100720020006600f600720020007000e5006c00690074006c006900670020007600690073006e0069006e00670020006f006300680020007500740073006b0072006900660074002000610076002000610066006600e4007200730064006f006b0075006d0065006e0074002e0020005000440046002d0064006f006b0075006d0065006e00740065006e0020006b0061006e002000f600700070006e006100730020006d006500640020004100630072006f0062006100740020006f00630068002000520065006100640065007200200035002e003000200065006c006c00650072002000730065006e006100720065002e> /ENU () >> >> setdistillerparams << /HWResolution [600 600] /PageSize [612.000 792.000] >> setpagedevice

Comments are closed.