[User] Please transcribe this text

[Assistant] Yes, well actually, you know, a lot of times now men, well they might have like a briefcase and an extra bag or a coat. You're actually not only supposed to have one bag but just one thing. A lot of research has shown.


[User] Please transcribe this text

[Assistant] Parents who have kids who are interviewing share these tips too. So uhm. Yea, just one bag.


[User] Please transcribe this text

[Assistant] Leave it in the car, leave it in the car if you can. Take off your scarf and mittens and put them away before you go in. If the front desk asks you, hey can I take your coat? The answer is always yes. You also say don't forget the back of your shoes.


[User] Please transcribe this text

[Assistant] Actually, the more powerful the person, the less they smile. And we also perceive this. So as women tend to smile more... Yeaah.


[User] Please transcribe this text

[Assistant] People will advise nervous interviewees, just smile if you're nervous. It actually is not a good thing to do, so we're going to talk a little bit later about what body language you can do.


[User] Please transcribe this text

[Assistant] And they'll cross their legs tightly to be as small as possible. That actually makes them look weak. Men do the opposite. Men to try to, when they're nervous, they try to gain territory. So they put an arm over a chair or they spread their legs sort of kind of wide when they're sitting. That is very aggressive for a female interviewer and for a male interview it's seen as competitive.


[User] Please transcribe this text

[Assistant] So for a man, try not to, you know, claim territory, plant your feet right, shoulder length is always good. And for a woman, don't tuck yourself in, you know, sort of claim your space, be a little bit more confident. It's a very easy thing you can do to show confidence.


[User] Please transcribe this text

[Assistant] Right, so uhm, actually this, the way that we're sitting right now, slightly angled, is the best way to talk to one another. There was a great study that was done that showed that when you are directly across from someone, whether that's a round table or a square table, you recall less of what they say.


[User] Please transcribe this text

[Assistant] There's something about it that we feel like we're adversaries. So if you come in and you have a choice of chairs in an interview, you know, to sit slightly angled right at the desk or even turn your body at an angle, that actually makes you look like you're on the same side. It subconsciously shows that we're on the same side, we're not adversaries.


[User] Please transcribe this text

[Assistant] I think, uh, this is, uh, Aussie weather. It's good for the cricket.


[User] Please transcribe this text

[Assistant] It's amazing because it's a club that helped put me on the map in the Premier League. Give me my opportunity, Bill Kenwright, David Moyes.


[User] Please transcribe this text

[Assistant] Uh, it was at a time when they signed me, uh, two days I'm in the club, two days later, Wayne Rooney's off to Man United and there was a bit of fanfare there and it was probably one of the best years we had in David Moyes' reigns.


[User] Please transcribe this text

[Assistant] I think firstly his technique is training. I'm going in the shade. I think firstly his training is techniques.


[User] Please transcribe this text

[Assistant] You know, making sure that you can header ball cleanly. The main thing is now with me going into coaching is speaking to younger kids is the goal never moves.


[User] Please transcribe this text

[Assistant] And you find chemistry between players. So I had Hippo.


[User] Please transcribe this text

[Assistant] I knew that the percentages of a cross from the second half of the goal to the back post was more percentages for me. You never really see me go near post because the percentages of it going over my head. Um, the other thing as well is timing of run.


[User] Please transcribe this text

[Assistant] So before every game, uhm, analysis was important that I knew who the weakest center back was. It was, you know, couldn't turn off his left shoulder better than he could his right. Or playing on the left back, whether it be, I don't know, maybe Ashley Cole or...


[User] Please transcribe this text

[Assistant] Ivanovich on the other side, I'll pick Ashley Cole as the set. So little things in the game, I think you have to go into all the detail of then being smart enough to use your power tool, which was heading. But, but I had a high fitness caliber that I would keep making the run into the box and a lot of goals will be scored later on.


[User] Please transcribe this text

[Assistant] Not only being explosive, I wasn't the quickest, I wasn't someone that shot outside the box, but what I did is I played to my strengths.


[User] Please transcribe this text

[Assistant] Whether it was Distan, Fellaini, Jagelka, uh, all these players, Karsly.


[User] Please transcribe this text

[Assistant] Yeah, I like being marked because that defender, uh, can only watch the ball. So for me, I like to...


[User] Please transcribe this text

[Assistant] I think when you look at styles of play, I think you look at philosophies in the way people play. If you look at the way Manchester City play, it's runs in behind, it's plays in the pockets, uhm, and then once they get in that danger, it's a fizzball along the floor. The percentages is, is, the defender misses it, he scores an own goal or gives away a corner or a throw in.


[User] Please transcribe this text

[Assistant] Uhm, there are the dinks you seen yesterday with Alonso into, uh, Alonso into, uhm, Abraham. Then you see again, uh, other goals that have been scored. Right, Edda. Yeaah.


[User] Please transcribe this text

[Assistant] You know, it's one of those that, I don't know if it's the art, I think it's just the philosophy of the football team, but uhm, you know, England's renowned back in the day, crosses second balls, headers. Yeah, and it's changing a little bit, but uhm,


[User] Please transcribe this text

[Assistant] You know, I think for me, when you look at games in general, uh, I like looking for the third man runner, the deep lying midfielder, whether it's one of the tens or a holder, making the running from behind and ghosting in because you want to see more midfielders breaking in from midfield and scoring, you know, goals.


[User] Please transcribe this text

[Assistant] Yeah, and I really love watching him progress as a manager, even though it's really early, because uhm, even with Mason Mount, he's instilling values of him as a player, and inside his players, to believe in them, to let them go and be free. So uhm, yeah, I'm really liking watching that.


[User] Please transcribe this text

[Assistant] And know that if the 18 yard box is bobbly, then I definitely want that ball in the air. Where I want the cut back, how I want it played to me. And then again, you're just thinking, you know, I always visualize goals. I always visualized, you know.


[User] Please transcribe this text

[Assistant] It's starting in the apprenticeship level with the academy and really using TV as an analysis, um, process to learn and understand the game at the top level, not only as a footballer, but now as a fan.


[User] Please transcribe this text

[Assistant] Obviously couldn't have been a rugby player. Of course you could. Nah, this physique's still 75, 76 kilos. Look, growing up, my mum's from Samoa, heavy culture in rugby, rugby union, rugby league, a lot of my cousins played here in the super league also, in the NRL, played with my cousins.


[User] Please transcribe this text

[Assistant] I had a big heart and playing with them I would try and run through brick walls but always be the kid crying at the end of the games and I had to change the game. Uhm, and I think it helped me mentally growing up with my cousins and, and, you know, always dusting yourself up and going again because uhm,


[User] Please transcribe this text

[Assistant] Coming to England as a 16 year old and understanding how difficult it was back there to come over, to earn a contract, to train twice a day and to even just to be seen, uhm, in training.


[User] Please transcribe this text

[Assistant] Was a big thing for me and that was, uh, you're probably one of the best things, but rugby is a big part of my, my, um, my upbringing and learning their traits of respect.


[User] Please transcribe this text

[Assistant] You know, humbleness and always, um, you know, off the pitch, you know, shaking hands after the battle on the pitch.


[User] Please transcribe this text

[Assistant] Us to get them bro? You trust us with such an important job? That's exactly the point.


[User] Please transcribe this text

[Assistant] First you must put the empty milk bottles in the pot. Really? What could all that do? GIVE THEM TO ME! Okay, relax.


[User] Please transcribe this text

[Assistant] Uuuh. To perish on the field is the greatest honor!


[User] Please transcribe this text

[Assistant] The operation's complete. My troop's turn is over.


[User] Please transcribe this text

[Assistant] Don't forget, I have the power of overkill.


[User] Please transcribe this text

[Assistant] So you're devoted to your strategy till the end. Very well. It's my troop's turn.


[User] Please transcribe this text

[Assistant] Is there a connection between magnetic center and consciousness?


[User] Please transcribe this text

[Assistant] In considering likes and dislikes, how can one separate what is real and what is false personality?


[User] Please transcribe this text

[Assistant] And if you'll notice, we even found a way of converting the roof garden into a landing field for helicopters. We'll be the only yacht club in the world with anything like it. We had a new application for membership this morning too. Colonel Rittenhouse. Of British Intelligence?


[User] Please transcribe this text

[Assistant] Well, even the British intelligence has to relax. You know him, don't you, Rick? Shipboard acquaintance. I cross with him on the Queen Elizabeth. How many memberships have we sold so far? Oh, I don't know exactly, but I'd say... In money, I mean. Well, I think it's about 250,000, isn't it, Rick? About that.


[User] Please transcribe this text

[Assistant] I'm only asking for an explanation, gentlemen. Which I don't think we should be put in the position of having to give.


[User] Please transcribe this text

[Assistant] In that case, you leave me no other choice but to call the police. Alright, have it your way. Bring on the cops and let them tag all of us. What do you mean?


[User] Please transcribe this text

[Assistant] This is a fraud. I'll get the best lawyer in the country. And the worst kind of publicity.


[User] Please transcribe this text

[Assistant] We're partners remember? We took it in, you give it back. You mean pay them back with my money?


[User] Please transcribe this text

[Assistant] What's the matter? Is it caliphate? Simple business law, Van. A partner's responsible for the actions of his associates. You don't have a paragraph to stand on.


[User] Please transcribe this text

[Assistant] Sounds like L.A. L.A.? The tower is Mission City. Mission City? A little haven of retired millionaires, just outside of Pasadena. Hasn't been touched yet. Maybe there's a reason. Could be the city fathers are too tough to handle. It doesn't matter. Since when? Since I decided the city manager's going to cover for us. Oh, ambitious, I don't know. That's practical. The old boy's got a daughter he's nuts about.


[User] Please transcribe this text

[Assistant] Simple. Until you try involving the girl, then maybe it's not so simple. Well, that's where you fit in, Rick. Unless, of course, you've lost that fatal charm of yours. I haven't had any complaints lately.


[User] Please transcribe this text

[Assistant] Her name is Deborah Owens Clark. 24 years old. Strictly respectable. Lost her husband in the war. Still hasn't gotten over it. In that frame of mind, she's a perfect mark for you. What's the fix? The same kind of a game we played with Vandalin.


[User] Please transcribe this text

[Assistant] Only this time, instead of raising money for a yacht club, we do it for a phony war memorial. A phony war memorial? That's a very patriotic idea. The details are all in this envelope. You and Duke memorize the stuff. You're leaving tomorrow morning. Well, in that case, we'd better go and pack. You're big enough now to sit on a suitcase by yourself, Duke. I want to talk to Rick alone.


[User] Please transcribe this text

[Assistant] I'm sending her to Havana. She'll like it there. That's what the travel folders say. But somehow I have an idea she prefers you to Cuban Rumba. Sookie, don't let your imagination run away with you. She's your girl. If you don't trust her, get rid of her. It's a funny thing, Rick. Tory's like a high tension wire. Once you grab on, you can't let go. Even if you want to. And I don't want to.


[User] Please transcribe this text

[Assistant] Well I hope you both will be very happy. But leave me out of it. That's the way I'd like it too.


[User] Please transcribe this text

[Assistant] I'm gonna double your take if you make good in California. You're generous. Very generous.


[User] Please transcribe this text

[Assistant] Gotta take it easy, Tori. Silky suspicious enough as it is.


[User] Please transcribe this text

[Assistant] After this next deal, I'm breaking away. I'm as fed up with silky as you are.


[User] Please transcribe this text

[Assistant] I promised a buddy of mine if I ever got by this way I'd drop in and see his family.


[User] Please transcribe this text

[Assistant] I wouldn't be too curious about who took it, Charlie. I think the boy's sorry.


[User] Please transcribe this text

[Assistant] I do know one thing, however. He was the last person on his mind before he went into battle.


[User] Please transcribe this text

[Assistant] Yes, he did. We were standing on the rail of an LST heading to the breakers.


[User] Please transcribe this text

[Assistant] Next to you, he's always talking about kids. How there were no bad kids, only bad breaks.


[User] Please transcribe this text

[Assistant] Something big and comprehensive, library, swimming pools, hobby rooms, all on the one roof. Sounded kind of far-fetched at first.


[User] Please transcribe this text

[Assistant] I'm a spy. I'm trying to find out what the reward for clean living is. Bet you loads of fun on the date.


[User] Please transcribe this text

[Assistant] Sure I know, but it takes more than just the right kind of a spirit to start something like that. It takes money.


[User] Please transcribe this text

[Assistant] The way you look right now, you can do it all by yourself. I think it's a wonderful idea, Deb. And quite a tribute to Jim. Kind of like, uh, a war memorial.


[User] Please transcribe this text

[Assistant] Read the sign. There's a rule against bringing women in this place. There's also a rule against intoxicating beverages. All right. Now look in the closet in the upper right.


[User] Please transcribe this text

[Assistant] Yeaah. I'm just trying to decide where to break it.


[User] Please transcribe this text

[Assistant] I can't take the chance on coming out here. Stokey's probably got Max on my tail right now.


[User] Please transcribe this text

[Assistant] A ship tourie to Havana. She winds up in Mission City. Who sent for her, Rick? How should I know?


[User] Please transcribe this text

[Assistant] Great. Until you bust it in. I said I'm sorry, but I'm not gonna write it on the blackboard a hundred times. I've seen the girl twice. Once with her father. We start looking at property tomorrow. Fast work. But don't sacrifice quite.


[User] Please transcribe this text

[Assistant] Silky can't see straight when he's got Tory on his mind. You seem to have the same kind of trouble. Sick and tired of the whole stupid setup. How'd you like a break with Silky? You think you're smart enough to operate without him? What do you think?


[User] Please transcribe this text

[Assistant] And you know what it means to be in combat week after week, month after month, tired, wet, lonely. You get to know a lot of people you never even met. You read the other fellow's mail when you don't get any and you talk for hours about his family and friends. Jim talks so much about fish eye that I felt I knew him a lot better than most people I really do know.


[User] Please transcribe this text

[Assistant] You've got Fifth Avenue written all over you. How are things in Mission City?


[User] Please transcribe this text

[Assistant] And directly behind that there's an open space here that could be converted into a baseball diamond. And then of course there's your woods for barbecues and campfires. Can't be that kind of planning, Deb. The architect was God.


[User] Please transcribe this text

[Assistant] We better get back. Santa Claus is waiting.


[User] Please transcribe this text

[Assistant] Price, a hundred thousand dollars. That'll take a lot of contributions.


[User] Please transcribe this text

[Assistant] Should have been. It's the last day of the year. Say, Deb, I've got an idea. Let's go out and celebrate tonight.


[User] Please transcribe this text

[Assistant] You'll have to start meeting people someday, if only to get contributions.


[User] Please transcribe this text

[Assistant] Funny, the sharpshooter's medal was lost in the mail while Jim was still in the States. Deb almost got sick about it. Although it's really the least important of them all. Sure. They give those away in popcorn boxes.


[User] Please transcribe this text

[Assistant] Yes, I know. It's always that way. But it's not their fault, it's yours.


[User] Please transcribe this text

[Assistant] They remind you of Jim, the times you had, the friends you made. And you let her get you down. I can't see it that way myself.


[User] Please transcribe this text

[Assistant] All these memories that are making you so miserable, they were happy ones, weren't they? And you're changing them into something else. Why not save them like you did Jim's medals and let them enrich your life instead of embittering it?


[User] Please transcribe this text

[Assistant] He has cattle interests all over the country and I just kind of wander around, keep an eye on him. I can't seem to settle down.


[User] Please transcribe this text

[Assistant] We'll buy a watch, family heirloom. Real diamonds in it too.


[User] Please transcribe this text

[Assistant] Can I cut in? Later son, we just met.


[User] Please transcribe this text

[Assistant] This is the California I heard about. Beautiful women lounging around swimming pools.


[User] Please transcribe this text

[Assistant] On the contrary, she was looking at me. Haven't you noticed? I have one brown eye and one blue one.


[User] Please transcribe this text

[Assistant] Looks like Vanillin's gonna be around for a while. That's what I thought.


[User] Please transcribe this text

[Assistant] When do we clear town? As soon as we get some dough. But the fix is off, Silky. I told you, Deb wants to do the whole thing with her own money. That's a one-way deal from her purse to McNulty's safe. We couldn't make a nickel out of that with a four-cent start. You give up too easily, Rick.


[User] Please transcribe this text

[Assistant] And we can be out of town before Vandalin has a chance to get the wrinkles out of his clothes. I don't think Deb would go for it. Well, you've talked her into everything else. You'll talk her into this.


[User] Please transcribe this text

[Assistant] What's eating you? Nothing's eating me. I don't like the play. It's too risky. Well, making a hundred grand always is. Or have you found something more practical than money?


[User] Please transcribe this text

[Assistant] I should have known. New Year's Eve, you were falling all over.


[User] Please transcribe this text

[Assistant] What's holding you back? Give him a break, Silky. He's never let us down yet. Don't front for me, Duke. If he doesn't trust me, he can get himself another boy. Get this straight, Rick. I spent a lot of money on this fix. We'll take what we can get.


[User] Please transcribe this text

[Assistant] You're going through with it hot or cold. You've got no lease on my blood supply. I'll quit any time I feel like it. Maybe you're like now. I didn't say that. But you meant it.


[User] Please transcribe this text

[Assistant] I got a girl staked out in the real estate office who thinks I stepped out of a dream. Good boy, Rick. I want you to handle the other half of the play, Duke.


[User] Please transcribe this text

[Assistant] How you gonna get McNelty out of the way? It'll be a cinch. He'll be in Arcadia showing me some property.


[User] Please transcribe this text

[Assistant] I gave you three minutes to put on your face and get over here after I passed your office. It's taking you four.


[User] Please transcribe this text

[Assistant] Only in the larger cities. How'd you like to clear out of this town?


[User] Please transcribe this text

[Assistant] That's too bad because if you did, you'd have a free ride clear to LaGuardia Field.


[User] Please transcribe this text

[Assistant] 9.30 tonight. I'll follow you in a couple of days.


[User] Please transcribe this text

[Assistant] Oh, well I just didn't like the idea of them pushing you into anything.


[User] Please transcribe this text

[Assistant] Alright, I'll be right over. Don't do anything till I get there. Bye.


[User] Please transcribe this text

[Assistant] Fine. Deb, uh, let me do the talking, will you?


[User] Please transcribe this text

[Assistant] Thank you. I abhor the idea of rushing people into a thing like this, but as a matter of fact, this sale is very important to me.


[User] Please transcribe this text

[Assistant] As I was explaining to Mrs. Clark, this is the first big deal I've handled since I started working with Mr. McNulty. I realize that, Mr. Swanson, but don't you think you could give us a couple more days to think it over? Well, I wish I could say yes, but this other client is ready to close. I don't think I could take the chance.


[User] Please transcribe this text

[Assistant] Deb, I tried to explain to you last night why I thought you shouldn't buy a Skyview with your own money. But I'm afraid I didn't tell you the most important reason.


[User] Please transcribe this text

[Assistant] If you buy Skyview, there'd be a lot of people who'd say it was just a stunt to drum up votes for the next election. Oh, no, Rick. I don't want this to sound as if I were interfering, but couldn't you buy the place on the QT, so to speak? Not if the check is made out to the McNulty Real Estate Office.


[User] Please transcribe this text

[Assistant] Fine, I'll be at your house tomorrow at 8.30 to pick up the cheque. Then I'll have the contracts drawn up and you can put them and the cash into escrow.


[User] Please transcribe this text

[Assistant] She trusted me. Just a wide-eyed kid who couldn't tell a beggar from a burglar.


[User] Please transcribe this text

[Assistant] If you could buy a cheap horse, you'd rent your mother out as Lady Godiva.


[User] Please transcribe this text

[Assistant] Theb wouldn't go for it. I told you, Jim was a ten god.


[User] Please transcribe this text

[Assistant] I didn't plan it, Duke. It just happened. Doesn't it always? What about the cheque?


[User] Please transcribe this text

[Assistant] I'll explain to Deb later. Sorry, Duke. That's all right. I just didn't know there was that much love left in the world. Silky's not going to like this. Silky's not gonna know if you'll help me. And I just gave you my share of a hundred thousand dollars. Isn't that enough? When Silky calls Stalem, tell him the fix is off until tomorrow. That'll give me a chance to clear town. You have as much chance of survival as a fan dancer with a broken arm.


[User] Please transcribe this text

[Assistant] Catch a train this afternoon for Frisco, be married there and board the Clipper for Honolulu.


[User] Please transcribe this text

[Assistant] I told you let me handle it my way. Anyway, Silky's plenty hot at me for curling the fix. Might have to start being a good boy for a while.


[User] Please transcribe this text

[Assistant] Hello, Mr. Owens, this is Rick. Look, something terrible just happened. No, I can't tell you about it over the phone. It's about Deb. Meet me right away at Ocean Drive, 2265. Right.


[User] Please transcribe this text

[Assistant] Deb. Deb! She'll be alright, she tripped running out.


[User] Please transcribe this text

[Assistant] Rick, is she dead? Yeaah. The woman was blackmailing Deb. I suppose we should have told you. She was Jim's girlfriend while he was in the army. She threatened to start a scandal unless Deb paid off.


[User] Please transcribe this text

[Assistant] Rick, what are we going to do? We'll think of something.


[User] Please transcribe this text

[Assistant] Who's the other dame? My daughter. She's out cold. She killer? We just got here ourselves. Have you called the cops? Not yet.


[User] Please transcribe this text

[Assistant] You better call them. But we don't know what happened. They'll find out. Wait a minute. What did you want the other girl for? Blackmail. Every kind in the books. Well Deb never hurt anybody in her life.


[User] Please transcribe this text

[Assistant] What I mean is this could all be a big mistake, but if you call the police, they'd be sure to arrest her. There's no other way. There must be another way. If that girl's a blackmailer, she must have had dozens of enemies. Any one of them could have done it.


[User] Please transcribe this text

[Assistant] Are you implying that this girl was framed? Why not? With that kind of circumstantial evidence, you wouldn't stand a chance.


[User] Please transcribe this text

[Assistant] If you could just forget you ever saw us. That's suppressing evidence.


[User] Please transcribe this text

[Assistant] Give us a break. We'll get the girl out of the way. No one will ever know. But we'd be accessories. Mr. Owens is city manager. I'm sure he'd cover for you. I'll do anything to keep my daughter from having to go through this. Will you help us? You're putting us in a tough spot, Mr. Owens. If we did, I'd have to have some sort of guarantee that you'd


[User] Please transcribe this text

[Assistant] No, it's a wonderful idea. But why didn't he ever tell me about it? Or write it in his letters?


[User] Please transcribe this text

[Assistant] And do you know that statistics prove that 75% of all adult criminals were formerly juvenile delinquents?


[User] Please transcribe this text

[Assistant] He wanted me to go on doing things for both of us. You told me that yourself.


[User] Please transcribe this text

[Assistant] Oh, so do I, Rick. But don't you think it's a little too expensive?


[User] Please transcribe this text

[Assistant] There are a lot of women in this town, Rick, who'd rather help kids than buy silly little hats.


[User] Please transcribe this text

[Assistant] I don't know, Rick. It's been so long since I've been out with anyone except Jim.


[User] Please transcribe this text

[Assistant] You're interested in beautiful women, aren't you Rick? I saw you looking at that woman in the real estate office.


[User] Please transcribe this text

[Assistant] No, I mean more than just helping to build it with other people's money. I want to build it all by myself with my own money.


[User] Please transcribe this text

[Assistant] I can afford it. My mother left me some money when she died.


[User] Please transcribe this text

[Assistant] Oh, after it's all built, there'll be so much to add. Well, they can help then. It's just that I want to move on and not have to wait for contributions.


[User] Please transcribe this text

[Assistant] There is. Will there be so much planning and figuring to do? And you're so good at that sort of thing. Oh, can't you manage to stay on? Well, at least until everything is finally worked out.


[User] Please transcribe this text

[Assistant] Well I'll make it out to you then, and you can endorse it when you get down there.


[User] Please transcribe this text

[Assistant] Hello? Oh, this is Mrs. Clark speaking. Would you read it to me, please?


[User] Please transcribe this text

[Assistant] You dropped your keys in the hall. I called, but you'd driven away. I followed you. Thought you might need them.


[User] Please transcribe this text

[Assistant] He would be the physical metaphor for the irresistible force. He just kept coming and throwing that dreadful overhand right that just hammered everybody into submission.


[User] Please transcribe this text

[Assistant] It was a fighter who gave you every ounce of energy he had every split second he was in the ring.


[User] Please transcribe this text

[Assistant] Well, the only other person I knew that was that clumsy when he started out and forever for that matter was Rocky Graziano. Both of them, their footwork was terrible.


[User] Please transcribe this text

[Assistant] He'd stuff his cash under his mattresses, hide it in toilet bowls, refrigerators.


[User] Please transcribe this text

[Assistant] It was on a four-lane highway and goes back through the oncoming traffic, weaving in and out. And naturally I've got my head down between my legs and I'm dying.


[User] Please transcribe this text

[Assistant] He absolutely disdained any danger of any form. He had that feeling of invincibility. He thought he was bulletproof, that nothing could hurt him.


[User] Please transcribe this text

[Assistant] Police officer. Italian. Went crazy. Grabbed Rocky. He was crying too. He made a signal. Another cop. 150 yards up. He picked it up. Another Italian. He grabbed Rocky. Hugging and kissing.


[User] Please transcribe this text

[Assistant] Rocky, what round are you gonna knock him out? He always said this, I hope to win.


[User] Please transcribe this text

[Assistant] He never mentioned anything about a knockout what round.


[User] Please transcribe this text

[Assistant] Oh, she was the boss all the way. She would holler at him, and he would pay attention too.


[User] Please transcribe this text

[Assistant] We had a whole bunch of Italian boys on Federal Hill. Massiano would throw the punch. And as he's going down, the boys on Federal Hill holler out, Timber!


[User] Please transcribe this text

[Assistant] He had what I always said to him all the time, you have long pockets, short hands.


[User] Please transcribe this text

[Assistant] knew about because had his own technique, his own way of tearing you up in close.


[User] Please transcribe this text

[Assistant] He didn't want an uppercut and cut the nose. It was a rare type of a cut because it went up his nose.


[User] Please transcribe this text

[Assistant] Muhammad had been off, Rocky had been off forever, but he did a great job and Rocky came to me and said, Angie is a nice kid.


[User] Please transcribe this text

[Assistant] If it's half the season, then you're, I mean, your playoff hopes are pretty much, uh, dashed, I would say at that point, unless they do something ridiculous. And then you, then he comes back, and you know, then what do you do? At that point, you lost a year of his contract. If you can still maintain that year of his contract,


[User] Please transcribe this text

[Assistant] And count next year as his first year, which is a possibility from what I understand. Is that not true? Then you get a year off. We have that much more cap space next year because he, he was one of that ridiculous contract for a million dollars if it's his first year. So the more I think about this, the more I think that possibly the best case scenario is the entire year.


[User] Please transcribe this text

[Assistant] There was many times when he would get $5,000, $2,500, $3,500 for guest appearance, for speaking. And if he knew a fighter was down and out, he'd call him up. Say, listen, I got a spot for you.


[User] Please transcribe this text

[Assistant] But give Joe, walk out the, the spot. Give Isaac Charles the spot.


[User] Please transcribe this text

[Assistant] Thanks for watching. If you liked that, subscribe to our YouTube channel for all the latest videos. And if you didn't, subscribe anyway. It's free, who cares?


[User] Please transcribe this text

[Assistant] Why do you ask? Oh no, no reason. I just wondered.


[User] Please transcribe this text

[Assistant] The Desire of Ages, Chapter 83, The Walk to Emmaus. Late in the afternoon of the day of the resurrection,


[User] Please transcribe this text

[Assistant] Two of the disciples were on their way to Emmaus, a little town eight miles from Jerusalem.


[User] Please transcribe this text

[Assistant] These disciples had had no prominent place in Christ's work, but they were earnest believers in Him.


[User] Please transcribe this text

[Assistant] They had come to the city to keep the Passover and were greatly perplexed by the events that had recently taken place.


[User] Please transcribe this text

[Assistant] They had heard the news of the morning in regard to the removal of Christ's body from the tomb, and also the report of the women who had seen the angels and had met Jesus. They were now returning to their homes to meditate and pray. Sadly, they pursued their evening walk.


[User] Please transcribe this text

[Assistant] Talking over the scenes of the trial and the crucifixion.


[User] Please transcribe this text

[Assistant] Never before had they been so utterly disheartened. Hopeless and faithless, they were walking in the shadow of the cross.


[User] Please transcribe this text

[Assistant] But they were so absorbed in their gloom and disappointment that they did not observe him closely.


[User] Please transcribe this text

[Assistant] They continued their conversation expressing the thoughts of their hearts.


[User] Please transcribe this text

[Assistant] They were reasoning in regard to the lessons that Christ had given, which they seemed unable to comprehend.


[User] Please transcribe this text

[Assistant] As they talked of the events that had taken place, Jesus longed to comfort them. He had seen their grief.


[User] Please transcribe this text

[Assistant] He understood the conflicting, perplexing ideas that brought to their minds the thought, can this man


[User] Please transcribe this text

[Assistant] Jesus knew that their hearts were bound up with Him in love


[User] Please transcribe this text

[Assistant] And he'd longed to wipe away their tears and fill them with joy and gladness.


[User] Please transcribe this text

[Assistant] Which was a prophet mighty in deed and word before God and all the people. But the chief priests and our rulers, they said, delivered him to be condemned to death and have crucified him.


[User] Please transcribe this text

[Assistant] With hearts sore with disappointment and with quivering lips they added, We trusted.


[User] Please transcribe this text

[Assistant] On the day that followed the day of the preparation, the chief priests and Pharisees came together unto Pilate, saying, Sir, we remember that that deceiver said, while he was yet alive, after three days I will rise again. But the disciples did not remember these words. Then he said unto them, Oh, fools.


[User] Please transcribe this text

[Assistant] Ought not Christ to have suffered these things and to enter into his glory?


[User] Please transcribe this text

[Assistant] The disciples wondered who this stranger could be, that he should penetrate to their very souls and speak with such earnestness, tenderness and sympathy and with such hopefulness.


[User] Please transcribe this text

[Assistant] Often they looked earnestly at their companion and thought that his words were just the words that Christ would have spoken.


[User] Please transcribe this text

[Assistant] They were filled with amazement and their hearts began to throb with joyful expectation.


[User] Please transcribe this text

[Assistant] Christ performed no miracle to convince them, but it was his first work to explain the Scriptures.


[User] Please transcribe this text

[Assistant] Jesus showed the importance of the Old Testament as a witness to his mission.


[User] Please transcribe this text

[Assistant] Many professed Christians now discard the Old Testament, claiming that it is no longer of any use. But such is not Christ's teaching.


[User] Please transcribe this text

[Assistant] So highly did he value it, that at one time he said,


[User] Please transcribe this text

[Assistant] It is the voice of Christ that speaks through patriarchs and prophets from the days of Adam, even to the closing scenes of time.


[User] Please transcribe this text

[Assistant] It is the light from the prophetic past that brings out the life of Christ and the teachings of the New Testament with clearness and beauty. The miracles of Christ are a proof of His divinity.


[User] Please transcribe this text

[Assistant] But a stronger proof that He is the world's Redeemer is found in comparing the prophecies of the Old Testament with the history of the New.


[User] Please transcribe this text

[Assistant] Reasoning from prophecy, Christ gave his disciples a correct understanding of what he was to be in humanity.


[User] Please transcribe this text

[Assistant] In accordance with the desires of men had been misleading.


[User] Please transcribe this text

[Assistant] It would interfere with the correct apprehension of his descent from the highest to the lowest position they could be occupied.


[User] Please transcribe this text

[Assistant] Christ desired that the ideas of his disciples might be pure and true in every specification. They must understand as far as possible.


[User] Please transcribe this text

[Assistant] As every transgressor of the law must die if he continues in sin.


[User] Please transcribe this text

[Assistant] Open in their minds that they might understand the scriptures. The disciples were weary, but the conversation did not flag words of life.


[User] Please transcribe this text

[Assistant] And assurance fell from the savior's lips, but still their eyes were holden. As he told them of the overthrow of Jerusalem, they looked upon the doom city with weeping, but little did they yet suspect who their traveling companion was.


[User] Please transcribe this text

[Assistant] They did not think that the subject of their conversation was walking by their side.


[User] Please transcribe this text

[Assistant] For Christ referred to himself as though he were another person.


[User] Please transcribe this text

[Assistant] They thought that he was one of those who had been in attendance at the great feast and who was now returning to his home.


[User] Please transcribe this text

[Assistant] He walked as carefully as they over the rough stones now and then halting with them for a little rest. Thus they proceeded along the mountainous road.


[User] Please transcribe this text

[Assistant] During the journey, the sun had gone down and before the travelers reached their place of rest, the laborers in the fields had left their work. As the disciples were about to enter their home,


[User] Please transcribe this text

[Assistant] Had the disciples failed to press their invitation,


[User] Please transcribe this text

[Assistant] They would not have known that their traveling companion was the Risen Lord.


[User] Please transcribe this text

[Assistant] But if men are too indifferent to think of the heavenly guest, or ask him to abide with them, he passes on. Thus many meet with great loss.


[User] Please transcribe this text

[Assistant] They do not know Christ any more than did the disciples as He walked with them by the way.


[User] Please transcribe this text

[Assistant] The simple evening meal of bread is soon prepared. It is placed before the guest who has taken his seat at the head of the table. Now he puts forth his hands to bless the food. The disciples start back in astonishment.


[User] Please transcribe this text

[Assistant] They rise to cast themselves at his feet and worship him, but he has vanished out of their sight.


[User] Please transcribe this text

[Assistant] But with this great news to communicate, they cannot sit and talk. Their weariness and hunger are gone. They leave their meal untasted.


[User] Please transcribe this text

[Assistant] And full of joy immediately set out again on the same path by which they came, hurrying to tell the tidings to the disciples in the city.


[User] Please transcribe this text

[Assistant] In some parts the road is not safe, but they climb over the steep places slipping on the smooth rocks. They do not see. They do not know that they have the protection of Him who has traveled the road with them. With their pilgrim staff in hand.


[User] Please transcribe this text

[Assistant] Sometimes running, sometimes stumbling, they press forward, their unseen companion close beside them all the way. The night is dark.


[User] Please transcribe this text

[Assistant] But the sun of righteousness is shining upon them. Their hearts leap for joy. They seem to be in a new world. Christ is a living savior. They no longer mourn over him as dead. Christ is risen. Over and over again they repeat it. This is the message they are carrying to the soaring ones. They must tell them.


[User] Please transcribe this text

[Assistant] Giving talks at this, uh, series over probably the last 20 years and, uh, it's always fun to come back.


[User] Please transcribe this text

[Assistant] And we also want to include internal APIs, so if you're on a large software project, you, uh, will be using, uh, lower level features, uh, which are usually, uh, exposed to you as an API.


[User] Please transcribe this text

[Assistant] The key motivation for an API is so that you can have some software resources that you don't have to personally write that you can reuse from others.


[User] Please transcribe this text

[Assistant] APIs are surprisingly important and valuable. I didn't realize this when we started, but uhm, there are thousands of APIs. Just for the web, there's a site called programmableweb.com which lists over 17,000 APIs with like 100 new ones appearing every month.


[User] Please transcribe this text

[Assistant] Uhm, there's a consulting company called Apogee that says 77% of companies rate their APIs as a really important part of their business model. Uhm, and in fact the market for API web middleware, and this is just for the web, was $5.5 billion in 2014.


[User] Please transcribe this text

[Assistant] Uh, another way of recognizing the importance of APIs, I was just visiting Amazon.


[User] Please transcribe this text

[Assistant] And they mentioned to me that if you go to their financial statements, uh, most people think of Amazon as a marketing comp- as a store. Uh, in fact, the store, the Amazon store lost $174 million, uh, in the last six months.


[User] Please transcribe this text

[Assistant] AWS, which is their cloud platform, made 1.8 billion dollars. So really, Amazon is a web services company that happens to have a little store as a side business. And, uh,


[User] Please transcribe this text

[Assistant] And other companies are starting to recognize that this is a key, uh, part of their potential business. Uh, part of the evidence of that is Google actually bought the Apogee consulting company for $625 million, uh, last year.


[User] Please transcribe this text

[Assistant] So why should we use human centered methods for APIs?


[User] Please transcribe this text

[Assistant] Somewhat controversial, my group thinks that we can treat programmers as if they're humans.


[User] Please transcribe this text

[Assistant] Uhm, and therefore we can use techniques that we have as an HCI community established over the last 20-30 years as successful ways of understanding people, of making things that work better for people, and evaluating, uhm,


[User] Please transcribe this text

[Assistant] As kind of an analogy with UX, which everybody understands as an important thing to start optimizing. And,


[User] Please transcribe this text

[Assistant] APIs actually have the word interface in them and in fact they are the user interface between this body of code and the programmers who are trying to use it so we can, uh, think about how to deal with that.


[User] Please transcribe this text

[Assistant] I like this definition of programming. Uh, if you think of it, programming is basically the human activity of translating a task that you have in your head that you're trying to get the computer to do into a representation that the computer can understand. And our claim is that the closer


[User] Please transcribe this text

[Assistant] This, the smaller this transformation is, the closer the mapping, uh, the better off you'll be. And other people have called this similar things.


[User] Please transcribe this text

[Assistant] Uhm, so in HCI, we always say if the user interface doesn't work,


[User] Please transcribe this text

[Assistant] For the people, it doesn't actually work even if it doesn't have any bugs.


[User] Please transcribe this text

[Assistant] You can say the same thing about APIs even if the functionality matches what people expect if it doesn't actually operate as people


[User] Please transcribe this text

[Assistant] Uh, want then you can't, uh, then you should not count it as working.


[User] Please transcribe this text

[Assistant] There's been a number of reported examples where the incorrect use of APIs where people misunderstood how to use an API and used it incorrectly resulted in bugs and even really significant security holes in software.


[User] Please transcribe this text

[Assistant] Uh, as I mentioned, APIs are now a big business and it's starting to be recognized that


[User] Please transcribe this text

[Assistant] Uh, the usability of the APIs are key influencers of whether people are going to use your APIs or not. And so this translates into millions or billions of dollars for certain companies. And so APIs usability actually is an important business asset, business advantage.


[User] Please transcribe this text

[Assistant] Uh, another thing that's important to recognize is that people don't just use one API, they're bound to be using multiple APIs and they have to work together. And finally, uh, even if you don't have this really dramatic


[User] Please transcribe this text

[Assistant] Mistake in the usability of your API, uh, it can still be a pain, uh,


[User] Please transcribe this text

[Assistant] Uh, Andrew McBean from Google wrote this article where he talked about how people can have, uh, death from a thousand paper cuts. That just annoyance after annoyance in their APIs eventually makes them so sick of it that they give up and try something else. Uh,


[User] Please transcribe this text

[Assistant] HCI, human centered methods is actually quite a broad category. Most people think of human centered methods as


[User] Please transcribe this text

[Assistant] We've applied these methods across the entire life cycle of an API.


[User] Please transcribe this text

[Assistant] So, uh, at the early, uh, stage where you're trying to envision or plan, uh, we've used, uh, techniques such as contextual inquiries and interviews and surveys, lab studies, and something called data mining where you take a big repository of software and look through it to see what kind of patterns you can find.


[User] Please transcribe this text

[Assistant] And all of these help you understand what the real problems are.


[User] Please transcribe this text

[Assistant] At design time, we have a variety of methods to help you design your APIs after you've figured out what they should be doing.


[User] Please transcribe this text

[Assistant] And this includes a natural programming process that we invented in my lab.


[User] Please transcribe this text

[Assistant] We also use graphic and interaction design as well as paper prototyping.


[User] Please transcribe this text

[Assistant] Standard HCI methods that most people wouldn't imagine you can apply to APIs in fact work perfectly well.


[User] Please transcribe this text

[Assistant] Uh, then you, you know, go ahead and implement your APIs, and then you need to evaluate them, and we again use fairly conventional HCI methods, uh, heuristic analysis, uh, cognitive dimensions, uh, we do usability evaluations, and sometimes we even do A versus B studies of the APIs themselves. Uh, we haven't used these in my lab, but uh,


[User] Please transcribe this text

[Assistant] Uh, and Andrew McVean at Google has reported that they, the key techniques that they use are automatic analysis, so they have tools that will, uhm,


[User] Please transcribe this text

[Assistant] Automatically run through an API and find certain problems in analogy with the Lint tool for C.


[User] Please transcribe this text

[Assistant] And they also make heavy use of peer reviews or design reviews or code reviews to look at the APIs. And then finally, after release,


[User] Please transcribe this text

[Assistant] There are lots of different mechanisms you can use, uh, log analysis.


[User] Please transcribe this text

[Assistant] Error reports, feedback from users, and so forth.


[User] Please transcribe this text

[Assistant] Uh, so I mentioned that we invented the, what we call the natural programming elicitation technique in my lab. Uh, this was quite a long time ago we started doing this. And it's basically an idea to try and understand how your API users are thinking about the problem you're trying to provide an API for.


[User] Please transcribe this text

[Assistant] And so what we're going for is what are people's mental models? What are people's vocabulary? How do they talk about?


[User] Please transcribe this text

[Assistant] Uh, this particular design. And we tend to use blank paper tests, uh, where you just, uh, give people a piece of paper and say, imagine you had an API for X. What would you, how would you want to use it? What would be some of the objects, classes, methods, and so forth?


[User] Please transcribe this text

[Assistant] Uh, the trick is to figure out a way of prompting for this without giving away the answers.


[User] Please transcribe this text

[Assistant] So if we say, you know, what would be a good object for sending mail? People will say, send mail.


[User] Please transcribe this text

[Assistant] And then you really haven't learned anything because they just used the words you just told them to use. So, uh, it's really tricky to come up with ways of asking the questions that don't give away the answers and that's kind of a lot of the, uh, interesting research around this method.


[User] Please transcribe this text

[Assistant] But we have done things like understanding the API architecture, trying to understand the vocabulary of people.


[User] Please transcribe this text

[Assistant] Uh, one to use and also which methods go on which classes.


[User] Please transcribe this text

[Assistant] So uhm, I have a much larger scale project called the Natural Programming Project that we've...


[User] Please transcribe this text

[Assistant] We've had this name since around 1999, 1995, but I've actually been working, as James mentioned, in the area of making programming better since as far back as 78.


[User] Please transcribe this text

[Assistant] So the goal in general of the natural programming project is to make programming easier.


[User] Please transcribe this text

[Assistant] And the way we try to do that is by making it more natural. And we don't mean natural like natural language, so we're not programming in English or French.


[User] Please transcribe this text

[Assistant] Instead we're trying to understand how people think about it and make it closer to that. And we...


[User] Please transcribe this text

[Assistant] Obviously have to do studies to understand how people are thinking about it. Uh, we use these studies to guide designs, and we create new languages, new tools, and, uh, again, new APIs.


[User] Please transcribe this text

[Assistant] And then we iterate, iteratively evaluate and refine these.


[User] Please transcribe this text

[Assistant] And we target three classes of programmers. Uh, there's expert programmers, which is most of our work.


[User] Please transcribe this text

[Assistant] Uh, we also, to some extent, look at novice programmers, which we define as people who are trying to become expert programmers, so people who are in school, for example. And separately, there's a group we call end-user programmers, and these are people who


[User] Please transcribe this text

[Assistant] One colloquial way of thinking about it are people who program because they have to and not because they want to.


[User] Please transcribe this text

[Assistant] So most, uh, computer scientists enjoy programming, that's why they went into it, uh, software engineers and so forth, and they program because they want to, uh, but there turns out to be a really large number of people


[User] Please transcribe this text

[Assistant] In fact, uh, in a survey, uh, that we did way back in 2005 and, uh, Chris Scafidi actually updated the survey just this year.


[User] Please transcribe this text

[Assistant] It's pretty consistent that as many as 13 million people say they do programming at work,


[User] Please transcribe this text

[Assistant] Even though only 3 million of them are trained as professional programmers, so.


[User] Please transcribe this text

[Assistant] There are lots of different jobs that involve doing some kind of programming. For example, uh, you know, writing webpages, you might write some Javascript, uh, doing spreadsheets, you might have to program.


[User] Please transcribe this text

[Assistant] Doing uhm, database accesses, uh, doing data analysis, all sorts of different.


[User] Please transcribe this text

[Assistant] Uh, kind of tasks require you to do programming, and so there are lots of people who do this as part of their job, not necessarily because they, uh, want to.


[User] Please transcribe this text

[Assistant] And of course, uh, there are scientists and engineers as well. And so one of the key points of this is that if you're designing APIs or any other kind of tool,


[User] Please transcribe this text

[Assistant] Uh, for programmers, it's going to be used by many people who don't necessarily have a software engineering background, and so you really need to think about how other kinds of people are going to be approaching the tasks that you have available for them.


[User] Please transcribe this text

[Assistant] Another way I like to describe this is, uh, as a gentle slope system, uhm, and so this is a kind of a characterization I've been using for many years. Uh, the vertical axis is how hard it is to do something and the horizontal axis is how much you can do.


[User] Please transcribe this text

[Assistant] Uh, and uh, so thinking of Java, it's pretty hard to get started. Uh, so the y-intercept is quite high, there's a lot to learn. If you want to be a Java programmer,


[User] Please transcribe this text

[Assistant] But you can't do very much with Java until you start learning APIs. And so that's why there's this wall here that after you learn Java,


[User] Please transcribe this text

[Assistant] And you're really good at object-oriented programming. That's not enough. You have to learn how to use swing and the listener pattern and all sorts of special other...


[User] Please transcribe this text

[Assistant] Things one needs to know to use Swing effectively or the networking package and you have to learn sockets and all these other kinds of things you have to learn to learn its APIs.


[User] Please transcribe this text

[Assistant] So that has a very high ceiling, you can do anything you want in Java, but maybe we'd like to have a lower threshold, so people can get started easier. So the web is a good example of something with a lower threshold.


[User] Please transcribe this text

[Assistant] You can use Dreamweaver or templates from different manufacturers or WordPress or whatever to get started and you can go along pretty well.


[User] Please transcribe this text

[Assistant] And then all of a sudden you realize you need to learn CSS and HTML, and that's a wall that you'd have to overcome. And then after that, you want to make your page dynamic or do some data checking, and you have to learn JavaScript, which doesn't help very much until you learn a million libraries.


[User] Please transcribe this text

[Assistant] How to do this stuff, but you have to know how to coordinate these five or six different languages together.


[User] Please transcribe this text

[Assistant] Uh, this is machine learning. Same kind of thing, that there's a million little libraries you have to learn and have to figure out how to get them to all work together.


[User] Please transcribe this text

[Assistant] Uhm, and it's always a lot of fun to put your own systems on this chart and see where you think they fall with respect to the other ones.


[User] Please transcribe this text

[Assistant] But none of these are close to the ideal, which we draw like this.


[User] Please transcribe this text

[Assistant] Uh, basically the idea is that no matter where you are, you can get started right away.


[User] Please transcribe this text

[Assistant] Uh, there's not a big barrier to getting started with this API and everything you want to do that's more sophisticated, uh, only requires a little increment of learning. And so you don't get these barriers, these big walls.


[User] Please transcribe this text

[Assistant] Uh, you get, uh, this nice gentle slope all the way through, and you can keep going, uh, and do as much as you want. So this graph applies to APIs, it applies to programming languages, kind of any kind of tool that lets you do, uh, sophisticated, uh, development.


[User] Please transcribe this text

[Assistant] And the nice thing about this goal is that I've been using this chart for years and we, you know, don't really have much hope of actually achieving this goal. Uhm, so this means I don't have to worry about retiring, I can just keep


[User] Please transcribe this text

[Assistant] Started with Jeff Stylos's PhD thesis where he graduated in 2009. He's now a lead API designer at IBM.


[User] Please transcribe this text

[Assistant] And he interned at the API usability group at Microsoft in the Visual Studio group with Stephen Clark who kind of invented this area of API usability.


[User] Please transcribe this text

[Assistant] And his thesis was, Jeff's thesis was a lot about which programming


[User] Please transcribe this text

[Assistant] Uh, patterns turn out to be the most problematic and, uh, most usable. And we looked at learnability, error rates, preferences, and so forth. We studied four different, uh, three different patterns and, uh, also collaborated with SAP and then, uh, recently Google has funded us to do some follow-on research, which I'll tell you about as well.


[User] Please transcribe this text

[Assistant] So the, uh, Jeff's first study, uh, he called the Required Constructor.


[User] Please transcribe this text

[Assistant] A pattern or the create set call pattern. And so this is the difference between


[User] Please transcribe this text

[Assistant] Being able to create an empty object, so here we're creating a foo, uh, which is empty, and then we're filling it in later. So this is the create, set, call pattern. And that's as opposed to having a required constructor.


[User] Please transcribe this text

[Assistant] Uh, so in this case, when you create a foo, you have to provide all the data that foo needs, and then you're not allowed to change it afterwards. Or maybe you are allowed to change it, but uh, typically you wouldn't.


[User] Please transcribe this text

[Assistant] Uh, so if you know anything about Java, then a string class is immutable in Java. There's no way to change a string after you've created one. Uh, whereas most of the other classes, uh,


[User] Please transcribe this text

[Assistant] are mutable and you can, uh, whoops, uh, you can create, uh, empty ones and fill them in.


[User] Please transcribe this text

[Assistant] So it turns out, uh, in our study that everyone preferred.


[User] Please transcribe this text

[Assistant] Uh, worried about this because the immutability pattern, the second pattern is actually considered much more secure and much better for threading if you imagine you have multiple threads. Uhm, if you have an immutable object it's perfectly


[User] Please transcribe this text

[Assistant] Fine to share them amongst the threads, cause nobody's gonna muck them up for anybody else. Uh, and so this is a better...


[User] Please transcribe this text

[Assistant] Approach for security, and so recently we had a follow on research where we showed that if you really wanted immutable objects there was a effective way of doing that, which we presented at the Software Engineering Conference last year, but it was a language design, not an API, so I'm not going to talk about it anymore.


[User] Please transcribe this text

[Assistant] Uh, probably the most famous study that we did was on what's called the factory pattern. Uh, so, uh, normally when in object-oriented programming, when you want a new object, you say new object. So, uh, widget w equals new widget.


[User] Please transcribe this text

[Assistant] In the factory pattern, you're not allowed to do that. You're required to go off and find this other thing called the factory and ask it for a widget. So in this case you say f dot create me a widget.


[User] Please transcribe this text

[Assistant] Uhm, and so this means that when you're trying to do stuff with widgets, you also have to somehow figure out that you need this other class as well. Very common pattern. It's used a lot in Java and .NET and SAP, and if you go to books, they actually recommend using this pattern.


[User] Please transcribe this text

[Assistant] In contrast, we found it to be really problematic. The programmers found it very hard to use. Nobody ever expected it to be there. And when it was there, it had a measurable impact on performance. It was two to five times slower and more problematic to figure out how to use.


[User] Please transcribe this text

[Assistant] Uh, another study we did was on object place, uh, method placement. So, in any real API you're gonna have multiple classes that are required, multiple different groups of functionality, and you have to figure out as a programmer how to put them together.


[User] Please transcribe this text

[Assistant] And, uh, that is often, uh, very difficult. Uh, so here's an example in Java in the mail program. There's two classes that coordinate together. One's called the transport class and the other's called the message class.


[User] Please transcribe this text

[Assistant] And you, as an API designer, have to pick one to put the send method on. So which one would you pick?


[User] Please transcribe this text

[Assistant] Right, so usually at this point I ask the class or the audience, so how many people think


[User] Please transcribe this text

[Assistant] Okay, uh, four or five. And how many people think it'd be a better design to have the send on the message class? Nobody?


[User] Please transcribe this text

[Assistant] Two or three. And all the other people who didn't vote, well, if you were an API designer, you're really not allowed to abstain. Right? You have to make a decision.


[User] Please transcribe this text

[Assistant] This is not one of those things where you can just say, well, I don't care. I'm not going to put it anywhere. Right? Because you need a send method or else you can't send your mail. So,


[User] Please transcribe this text

[Assistant] You have to really, uh, make a choice. Uhm, and so it turns out in Java it was put on the transport method, uh, but everybody expected it to be on the send, on the message method.


[User] Please transcribe this text

[Assistant] Everyone expected to be able to take your e-mail message and send it somewhere, because that's kind of how you talk about it.


[User] Please transcribe this text

[Assistant] And the fact that it was in the wrong place actually was very confusing to people and they would go to the message class and they couldn't find the send method anywhere. So now they were really confused about how they should go about sending emails.


[User] Please transcribe this text

[Assistant] Uhm, and it turns out that this had a really significant impact on people's performance between 2 and 11 times slower as well as making it much harder to know what to do. Another interesting observation,


[User] Please transcribe this text

[Assistant] Is that the naming of classes is often very problematic. So in Java,


[User] Please transcribe this text

[Assistant] If you want to do something to the disk, you might think in your head that you want to deal with files. And if you go and search the class hierarchy for file, you'll discover it's an abstract class that you can't actually use.


[User] Please transcribe this text

[Assistant] Right? Uh, it's an interface or whatever. And so it'll say, oh, you can't use me, but I have 325, uh, you know,


[User] Please transcribe this text

[Assistant] Subclasses, pick one of those. So that's not particularly helpful. You have to know that you need to use a stream buffer, which is not anybody's first guess of the name of the thing you want to use. So, this kind of


[User] Please transcribe this text

[Assistant] Object-oriented design where you put this, the name everyone thinks of at the top.


[User] Please transcribe this text

[Assistant] Seems to make perfect sense from kind of an object hierarchy point of view, but from a usability point of view, uh, is really a nightmare.


[User] Please transcribe this text

[Assistant] So we, uh, leveraged all of this research into a really, uh,


[User] Please transcribe this text

[Assistant] Uhm, fruitful multi-year collaboration with a software company called SAP, which is based in Germany. Uh, it's actually one of the largest software companies in the world that nobody's heard of because they all will, they only do backend software. So, software that runs big corporations like universities and


[User] Please transcribe this text

[Assistant] Uh, so they definitely, uh, they definitely are problems with SAP software. Uhm, and uh,


[User] Please transcribe this text

[Assistant] Somehow they needed us to tell them what the problem was here. Uh, so this was, uh, back in the days before REST, everyone was using web services called SOAP, uh, which stood for Simple Object Something Protocol. It wasn't simple at all, which is why no one uses it anymore. Uhm, and this is their actual names of


[User] Please transcribe this text

[Assistant] And this is looking at their APIs in an IDE. And you'll notice the first six look identical.


[User] Please transcribe this text

[Assistant] Uh, in fact, there are differences way over here somewhere, and if you hover over them,


[User] Please transcribe this text

[Assistant] The differences are all in the middle of these long names.


[User] Please transcribe this text

[Assistant] And so, anyway, somehow they needed us to tell them that was a problem.


[User] Please transcribe this text

[Assistant] Uhm, their documentation was also problematic.


[User] Please transcribe this text

[Assistant] Uh, so we did a map of all the different parts of their documentation and what you had to do to navigate from the top down to finding the actual name of an operation that you wanted.


[User] Please transcribe this text

[Assistant] And this shows some of the paths of some of our, uh, users, uh, went and there were lots of cycles and dead ends and misleading parts of the documentation.


[User] Please transcribe this text

[Assistant] Uh, one interesting observation is that at some point the documentation changed color.


[User] Please transcribe this text

[Assistant] You notice that all the documentation has these things like SCM and SRM and if you're a business person, these make a lot of sense. If you're not, you don't know what these acronyms stand for. It turned out that non-business people had no hope of navigating this documentation and finding anything appropriate.


[User] Please transcribe this text

[Assistant] So, uh, what would you do with this information? If you know that APIs are problematic, ideally you would change them, but, you know, we as researchers are not going to have much luck changing the Java APIs. Uh, and also as an API designer, you may have other things that you're trying to optimize, not necessarily entirely, uh,


[User] Please transcribe this text

[Assistant] Uh, focusing on usability. It may have to be backwards compatible. It may have to have higher performance.


[User] Please transcribe this text

[Assistant] And therefore, uh, we can think about tools. So how can you use tools to fix some of these problems with the APIs? And so we built these seven tools, all of which have wonderful, uh, gemstone names and acronyms that you'll see and, uh, that, uh, help you with the, uh,


[User] Please transcribe this text

[Assistant] With these kinds of problems. So the first one is Micah. Making interfaces clear and accessible.


[User] Please transcribe this text

[Assistant] Uhm, and this was right at the beginning of the era where people recognized that Google was the, uh, super best way of finding documentation. And so whenever anybody had a programming problem, they would type it into Google and find the appropriate examples and documentation and, uhm, this was right at the beginning where that was starting to be done and


[User] Please transcribe this text

[Assistant] We recognize that this was kind of a totally new paradigm of...


[User] Please transcribe this text

[Assistant] In the return values, and you can hover over the different keywords and find which ones describe it.


[User] Please transcribe this text

[Assistant] Put this little icon near the results that had


[User] Please transcribe this text

[Assistant] Code examples, because everybody loves code examples, and many of these don't have them. And so, you know, this way you can augment the results that it gives back to make them more useful for programmers. And this obviously is a ubiquitous kind of thing that's done now, but we were really the first people to show it. And, uh, happy to say we recently won an award.


[User] Please transcribe this text

[Assistant] This was just two weeks ago, uh, and here's Jeff and me, uh, winning this award, uh, in October for this paper, it's a most influential paper. Jeff's getting grade two, that happens to, uh, students, they get older too.


[User] Please transcribe this text

[Assistant] Another, the next tool we built, uh, Java API documentation with extra information tacked on for emphasis.


[User] Please transcribe this text

[Assistant] Uhm, takes the Java doc and tries to make it more useful using some of this kind of knowledge, so.


[User] Please transcribe this text

[Assistant] Uh, one of the things it does is to make the names of the classes, uh, bigger based on how often they are used. And so this would guide people to using the right classes to start with because there's just otherwise a really long list.


[User] Please transcribe this text

[Assistant] Uh, remember the problem about where the send method should be? Uh, so this green thing is what we call a placeholder that in the message class you can put this pretend version of send which says, oh there really is no send here, go off to this other place and find it.


[User] Please transcribe this text

[Assistant] Uh, so the idea is to, you know, help people exactly where they need the help, uh, without trying to put it somewhere else, uh, that it doesn't actually exist.


[User] Please transcribe this text

[Assistant] And, uh, we also, uh, scoured the web for examples of how to use the class, uh, and provided those, and, uh, also showed which other classes tend to be used together. So remember you need to use the transport class with the message class, uh, and so we, uh, tell people about that.


[User] Please transcribe this text

[Assistant] So, wouldn't it be nice to have that information directly in the


[User] Please transcribe this text

[Assistant] Uh, IDE and the tool that you're using for editing your code. And so that's what Calcite does.


[User] Please transcribe this text

[Assistant] The idea here is that we just bring all that information into Eclipse. So while you're, everybody's favorite way of


[User] Please transcribe this text

[Assistant] Doing coding or exploring an API is by typing dot and looking at the popup autocompletes. And so we wanted to add more information into that popup menu.


[User] Please transcribe this text

[Assistant] And so that's what, uh, calcite does. Uh, in this case, in a typical, if you ever tried popping up the autocomplete menu after an equal sign, you'll discover not much happens. It's really pretty useless. Uhm, and that's why it doesn't come up by default. You know, it only comes up by default after a dot.


[User] Please transcribe this text

[Assistant] Uh, but you can pop it up anywhere you want and you can see what it will tell you. We augmented that menu with a lot of extra information like how to use sockets to get one of these objects to, uh, exist. How to use the factory pattern.


[User] Please transcribe this text

[Assistant] And it brings in a lot of the other information, uh, into the pop-up menus as well. So if you go to the message class and pop-up.


[User] Please transcribe this text

[Assistant] Uh, the menu of send will be there even though it doesn't really exist and it will tell you how to go off and find the real code.


[User] Please transcribe this text

[Assistant] Uh, jadeite and calcite were based on scouring the web for examples, and that obviously works really well for Java, but it won't work for most other tools that are poorly, you know, that aren't widely used or which aren't comprehensive. And so daysite, uh, is another kind of boring rock, uh, but it was an interesting system.


[User] Please transcribe this text

[Assistant] For design annotations for complementing interfaces targeting effectiveness was by a visitor, uh, who


[User] Please transcribe this text

[Assistant] Uh, came over and the idea of here is to use the Java annotation mechanism, which is surprisingly, uh, flexible. Uh, and eh, in Java annotations you can add an at sign.


[User] Please transcribe this text

[Assistant] And then put kind of anything you want and then you can run processors that compile time that will check things and generate data and so forth. So the idea of DaySight is you annotate things like factory patterns or helper functions and then, uh,


[User] Please transcribe this text

[Assistant] It will generate a plugin for you that can then be used in, uh,


[User] Please transcribe this text

[Assistant] By scouring the web, but which the developer knows they're doing and so this makes it a lot easier.


[User] Please transcribe this text

[Assistant] Uhm, I had a visitor from Germany who had a system in German called Orklus, which is German for Euclase, which is another gemstone.


[User] Please transcribe this text

[Assistant] He luckily made us an English acronym or we wouldn't be able to tell you what it was. And the idea of this is to deal with the common case that programmers like to copy and paste from examples.


[User] Please transcribe this text

[Assistant] And so what Orychlos does is it helps keep track of where you copied the code from and how it should automatically adapt it from the source to the destination so it fits in better with your destination.


[User] Please transcribe this text

[Assistant] And so it can do some static analysis in order to, uh, help you update the code to a better fit.


[User] Please transcribe this text

[Assistant] Uh, graphite, uh, is a cute little plugin that's, uh, to help you with constants, literal constants. Uh, it turns out there are a lot of special constants you need to do in languages, like if you want to color.


[User] Please transcribe this text

[Assistant] What kind of color should you use? Uh, and this will basically pop up a color wheel and let you pick the color like you would in a graphics editor, uh, but instead it'll just plop in the, into the code. And it also works for regular expression.


[User] Please transcribe this text

[Assistant] Associative perusing of APIs that identifies targets easily. Uh, the idea of this is to kind of look at the documentation sideways. It's kind of a new way of looking at documentation by association. And the goal of Appetite is that maybe you have the question, which classes have a read method?


[User] Please transcribe this text

[Assistant] And so you just click on read and that shows you all the classes.


[User] Please transcribe this text

[Assistant] Uh, that have a read method sorted by popularity.


[User] Please transcribe this text

[Assistant] So the most popular messages are post-mobile classes are bigger.


[User] Please transcribe this text

[Assistant] It also answers the question, what other methods are typically used with read, and they appear here, or which actions do read usually associate with, and so forth. And so by clicking on things in one column, you get a bunch of associations of different types in the second column. And then you can keep going as far as you want, and this will, uh, you know,


[User] Please transcribe this text

[Assistant] Provide information, uh, that's associated with each of the actions.


[User] Please transcribe this text

[Assistant] A former student who I was on their thesis committee went to Google and said, we really care about API usability. Wouldn't you like to start a new project studying modern API, modern web service APIs? And so that's kind of fun. And so the first two projects we did were to compare API style guides and interview a bunch of API designers.


[User] Please transcribe this text

[Assistant] So this, uh, style guidelines, uhm, we just presented, uh, Monday, uh, at the Plateau Conference in Vancouver.


[User] Please transcribe this text

[Assistant] And the idea of this is that a surprisingly large number of companies will publish guidelines. So you've probably heard of Java guidelines or the Apple style guidelines. So these are style guides for user interfaces, but they're also style guidelines for APIs.


[User] Please transcribe this text

[Assistant] And there's a surprisingly large number of them. Uh, we found 40 of them fairly quickly. And 32 of those were about


[User] Please transcribe this text

[Assistant] These REST APIs, the modern style of providing web services, uhm, and, uh, so we were wondering, you know, what people are saying about REST APIs and how much they agreed and disagreed.


[User] Please transcribe this text

[Assistant] Uh, so these are the 32 ones that we analyzed in depth in alphabetical order.


[User] Please transcribe this text

[Assistant] Uh, in the paper, it has a few, the full URLs of all of them if you want to see more detail.


[User] Please transcribe this text

[Assistant] The kind of, some of the big names are there, Amazon, Google, IBM, Microsoft, uh,


[User] Please transcribe this text

[Assistant] We found some of these in a website called apistylebook.com are the ones we just found with Google search.


[User] Please transcribe this text

[Assistant] Uh, it's also interesting to see the giant differences in the size of these. So the Google APIs are trying to be quite comprehensive and explain kind of all of the things you're supposed to do and not do.


[User] Please transcribe this text

[Assistant] Uh, as a Google API designer, uh, whereas other ones that are particularly short like, uh, rest cheat sheet are intended to be kind of read from cover to cover and just give you a sense of what sort of things you should be thinking about.


[User] Please transcribe this text

[Assistant] We looked at all the things that all the guidelines have said and collected their recommendations into categories. And so these are the, uh,


[User] Please transcribe this text

[Assistant] categories of guidelines that we found, uh, and they're sorted here by the number of different


[User] Please transcribe this text

[Assistant] Guideline sets that mention that particular category of, uh, problem. So, 30 of the 32 guideline sets mention status codes.


[User] Please transcribe this text

[Assistant] So the paper talks about most of these, uh, but here I'm just gonna talk about a few that we found particularly interesting. So all of the APIs, uh, have


[User] Please transcribe this text

[Assistant] You know, have to deal with versioning. You know, you have a new version and it works slightly differently. How do you tell API users what the version is?


[User] Please transcribe this text

[Assistant] These differed significantly across the different guideline sets. Uh, some had the typical three level.


[User] Please transcribe this text

[Assistant] Uh, versioning, like 2.4.1, where the last one was either a patch or a draft number. Some just said to use 2, 1,


[User] Please transcribe this text

[Assistant] Just said to have a single number, and one said don't use numbers at all, use dates instead.


[User] Please transcribe this text

[Assistant] They also had different locations where they wanted you to put the version. Uhm, 11 of them said it should be in the URL, like here.


[User] Please transcribe this text

[Assistant] Uh, eight of them said it should be in the HTTP header and not in the url number, url at all. And then one said you should put part of it in the url and the other part should be in the parameter in the body of the message. So, surprising amounts of differences there.


[User] Please transcribe this text

[Assistant] Backwards compatibility is really important, uh, so when you have an API


[User] Please transcribe this text

[Assistant] Uh, and you need to change it. Uh, should the change be done in a way that code that's using this API will not break? Uh, that's obviously better. Uhm, some of the guidelines mentioned what it means to be.


[User] Please transcribe this text

[Assistant] Uh, backwards compatible, what it means to be a breaking change or a non-breaking change.


[User] Please transcribe this text

[Assistant] And a number of them mention Postel's law, which is kind of a description of the web and web browsers, how they will try and be generous and accept kind of anything and try to deal with it. Uh, liberal in what you accept and uh, conservative in what you send out. So, uh, this is a good idea for APIs as well.


[User] Please transcribe this text

[Assistant] Uh, naming, uh, was interesting, uh, so the Google API has very specific recommendations for particular words to use for particular kinds of things, whereas virtually none of the other APIs, uh, picked particular words, although a lot of them had something to say about


[User] Please transcribe this text

[Assistant] Parts of speech, so use nouns for this kind of thing and use verbs for this other kind of thing. Because obviously in English you can mix those up, uh, and that gets to be confusing. So they said, you know, always use nouns for object names or something like that.


[User] Please transcribe this text

[Assistant] And then there's a lot of differences in how to, what kind of case to use, uh, like uppercase, lowercase, uh, camel case, I think we've all seen a lot in, uh, various kinds of code, uh, that's recommended by some people as opposed to snake case, fun name, uh, and that's where you use underscores.


[User] Please transcribe this text

[Assistant] Uh, and then for URLs, uh, where you, um, you're allowed to use, uh, hyphens. Apparently that's called kebab case, like a shish kebab through the middle. I never heard that, but that's fun too. Uh, so you can use kebab case or, uh, or snake case. And surprisingly, lack of consistency. They, these recommendations are all over the place, uh, with the different, uh, guideline sets. Everybody said documentation is good. No surprise there. Uh, and


[User] Please transcribe this text

[Assistant] Everybody's in favor of having good documentation, but there was controversy about whether the documentation should be automatically generated or not.


[User] Please transcribe this text

[Assistant] So web service APIs are basically about the information sent over the internet, but the consumers are in typically a wide variety of languages.


[User] Please transcribe this text

[Assistant] Uh, if you're talking about server-side, then again, there's a whole other collection of languages. Dart, Rust, uhm, Go.


[User] Please transcribe this text

[Assistant] So some of these, uh, some of the companies have to have the same API implemented for 10 or 15 different languages and that can be really difficult to manually create those. And so there are lots of tools that automatically will create your libraries and then create the documentation that goes with it. And so some of the guideline sets


[User] Please transcribe this text

[Assistant] We're in favor of doing that to make sure that they're consistent.


[User] Please transcribe this text

[Assistant] On the other hand, that documentation is known to be problematic, so some of the documentation, some of the guideline sets recommended against doing it automatically, and one said, well, do both. Have it automatically generated and then manually fix it, which kind of defeats the whole purpose.


[User] Please transcribe this text

[Assistant] Uhm, and one tool that's starting to be popular is called Swagger, which is a public tool that people are starting to use. It's also called OpenAPI.


[User] Please transcribe this text

[Assistant] So that was fun, but we really wanted to learn more.


[User] Please transcribe this text

[Assistant] Why people were making these recommendations and what designers were thinking about. And so we ran an interview study, uh, so far with 14 API designers across four companies. I'm not allowed to tell you who they are because that was confidential. Uh, we promised them confidentiality, but it was pretty extensive interviews with each one.


[User] Please transcribe this text

[Assistant] One and a half to two hours each, and we were able


[User] Please transcribe this text

[Assistant] Uh, and they were also obviously really experienced programmers as well, uh, mean of 16 years up to 28 years of programming. And then we had a fun little workshop that was kind of...


[User] Please transcribe this text

[Assistant] A meeting like this where we got to discuss API design with 50 different designers at the same time.


[User] Please transcribe this text

[Assistant] And we are in the process of trying to recruit a few more to interview.


[User] Please transcribe this text

[Assistant] And these people represented APIs in a wide variety of situations, uh,


[User] Please transcribe this text

[Assistant] Old-fashioned, in some sense, C++ APIs, Java, Dart, C-sharp, and of course many of the modern REST APIs as well.


[User] Please transcribe this text

[Assistant] And some were for internal use and most of them were external, externally visible.


[User] Please transcribe this text

[Assistant] Uh, how do you measure the size of something like this? And typically, uh, the answer they would come up with is how many methods they had, so 20 to 300 methods.


[User] Please transcribe this text

[Assistant] But another answer is how many users they had. So, uh, the kind of minimum was a hundredth of users and the maximum was millions of programmers using their APIs. So, obviously quite, uh, significant, uh, uhm, impact.


[User] Please transcribe this text

[Assistant] We asked them what it meant to be a high quality API, and the number one answer was simplicity.


[User] Please transcribe this text

[Assistant] Uh, typically meaning easy to learn, uh, only does one thing, does it very clearly, uh, people can figure out how to use it without reading the documentation very much. Uh, the number one goal, uh, to have a really high quality API.


[User] Please transcribe this text

[Assistant] Uh, the second thing people mentioned, which kind of goes along with it, is matching the user's needs, uh, which everyone reported was hard to identify, hard to figure out exactly what people wanted this API to do.


[User] Please transcribe this text

[Assistant] Uh, and how they wanted to express it. So that was their kind of number one


[User] Please transcribe this text

[Assistant] Uh, thing that they had difficulty finding out that would help them with their jobs.


[User] Please transcribe this text

[Assistant] They, uh, all, uh, recognized that consistency was a key goal, key problem that they had to solve, uhm, and hence this was driving a lot of the guideline sets, because that is a big way of achieving consistency.


[User] Please transcribe this text

[Assistant] We asked them what would be an example of a good API and, uh,


[User] Please transcribe this text

[Assistant] Uh, in terms of a conventional Java style API, people mention Guava as an exemplar of high quality API. And then for web services, typically people said Stripe, which is a financial processing API, uh, was


[User] Please transcribe this text

[Assistant] Held out as being high quality, well documented, easy to understand and simple.


[User] Please transcribe this text

[Assistant] Uhm, and interestingly we asked people how did they learn to do API design and virtually nobody had any training in this at all.


[User] Please transcribe this text

[Assistant] That nothing that anybody learned at university, even the ones who had PhDs in software engineering, nothing they learned was relevant.


[User] Please transcribe this text

[Assistant] To the task of API design. That was a little discouraging. Uhm, the key ways they learned was on the job. So more senior people said, oh yeaah, we just tried a bunch of things and we learned what didn't work and learned from our mistakes. Learned what people complained about and what ended up being hard to maintain. Uhm, more junior people said they were


[User] Please transcribe this text

[Assistant] Fortunate to have senior people around to learn from, to apprentice with, uh, many of the


[User] Please transcribe this text

[Assistant] Companies use design reviews, as I'll mention in a minute, and they found that also very illuminating when senior people came and looked at their designs and told them and gave them feedback about it.


[User] Please transcribe this text

[Assistant] Only one of all the people we talked to said that anything that was even mentioned in class was useful, and he happened to have a professor that mentioned API design and taught a few things about it in one lecture of a software engineering class. But other than that, nothing anybody learned.


[User] Please transcribe this text

[Assistant] Uh, was relevant, even though this is a quite a large number of people are involved in this kind of task as part of their work in companies.


[User] Please transcribe this text

[Assistant] And, uh, we asked them if there's some book or blog or...


[User] Please transcribe this text

[Assistant] You know, discussion forum where you typically look for good information and nobody really said anything.


[User] Please transcribe this text

[Assistant] Uh, and certainly there wasn't any consistency amongst different people.


[User] Please transcribe this text

[Assistant] So we were wondering how people organized their API design, what was the process that they used, and it was quite informal. Uh, you would imagine something as high impact as APIs might have a very strict or formal process, we did not find that. Uhm, many people mentioned having design reviews, so,


[User] Please transcribe this text

[Assistant] Uh, in companies it seems like code reviews are key, uh, quality, uhm,


[User] Please transcribe this text

[Assistant] Did these kind of reviews of early designs, but usually it was the finished implementation.


[User] Please transcribe this text

[Assistant] They, uh, as I mentioned, they were all these guidelines, uh, so you can imagine that the people who worked at the companies that had guidelines, uh, found those guidelines particularly useful as a way of trying to, uh,


[User] Please transcribe this text

[Assistant] Follow, uh, but they also reported that occasionally they


[User] Please transcribe this text

[Assistant] Well, we, they didn't actually admit this, but other people said that they weren't really following the guidelines and so that was sometimes a problem.


[User] Please transcribe this text

[Assistant] But besides that, there was really no standard process. There really was quite a divergence of the kinds of things that people would, uh, do and the process they'd use.


[User] Please transcribe this text

[Assistant] Uhm, I thought it was really interesting that, you know, if you think about regular GUIs, at this point, no major company is going to let their programmers design a user interface anymore. That they have UX specialists that do that kind of thing.


[User] Please transcribe this text

[Assistant] Uh, this was not the case for APIs. The LEAF engineers, the kind of lowest level engineers, were in fact empowered with making significant design decisions about the APIs.


[User] Please transcribe this text

[Assistant] There's some companies that had product managers that would sometimes get involved.


[User] Please transcribe this text

[Assistant] And one company mentioned the chief architect, which would deal with kind of architectural issues, but there still were many decisions that a leaf engineer would get to decide for themselves.


[User] Please transcribe this text

[Assistant] And a little bit sad to notice or to identify that nobody had ever done or even heard of doing a usability evaluation of the API before it was released. Uh, they pretty much relied on


[User] Please transcribe this text

[Assistant] Uh, beta releases, and so they would release it, and the beta re-


[User] Please transcribe this text

[Assistant] Users would give them feedback on the usability of their APIs and that was kind of it, uh, after they did the guidelines and the code reviews.


[User] Please transcribe this text

[Assistant] Uh, and also the documentation was typically written by the engineers themselves and some high impact APIs did have tech writers helping, uh, but generally it was the engineers themselves.


[User] Please transcribe this text

[Assistant] The key issue, key problem that people mentioned was backwards compatibility. Uh, they were always really worried about this, uh, because APIs with millions of programmers out there, you know, if you change something, people are going to be angry at you. And, uh, uh,


[User] Please transcribe this text

[Assistant] An interesting piece of feedback was that people working in old-fashioned languages like C++ said that pretty much anything they did was a breaking change. That there's no such thing as a non-breaking change in C++ because people have visibility kind of in any, everything and there's really no information hiding in something like C++.


[User] Please transcribe this text

[Assistant] We heard all sorts of stories about how, you know, people had unit tests that checked for the actual string of the error message.


[User] Please transcribe this text

[Assistant] And so when they changed the error message as a result of a refactoring, uh, then people's tests broke. Now the line number in the code that was exposed as part of the debugging message was being used by people. And so basically there was no, nothing that people could do that was a non-breaking change. And so that was really annoying. Um, one, uh, group of people had at their, uh,


[User] Please transcribe this text

[Assistant] Uhm, that uses their API, but it was related to me that it's less problematic than you think because they had good unit tests.


[User] Please transcribe this text

[Assistant] And so the claim was that the API designer could change the code.


[User] Please transcribe this text

[Assistant] Recompile it, run all the unit tests, and if it all passed, then...


[User] Please transcribe this text

[Assistant] They were golden and they didn't have to involve the actual authors of that part of the code. So that still seems a little worrisome to me. Uh, what are the other challenges?


[User] Please transcribe this text

[Assistant] And these are some of the things that we're working on for future research is what are the best design patterns for APIs? Do we have any way of talking about, uh, good things and bad things, especially for web APIs? What are the right ways to go? How do we help people coordinate multiple APIs? There's really been virtually no research.


[User] Please transcribe this text

[Assistant] On what you can do in terms of documentation or tooling to help people with putting APIs together.


[User] Please transcribe this text

[Assistant] And do we need to invent some new evaluation mechanisms? At lunch we were talking about


[User] Please transcribe this text

[Assistant] You know, how do you evaluate if this new version of your API is actually better or not?


[User] Please transcribe this text

[Assistant] Or is my API better than your API? How would you go about evaluating that?


[User] Please transcribe this text

[Assistant] And finally, if you're a company, what are the best practices in API design? If you wanted to, uh, say that you are following best practices, what would that even be like?


[User] Please transcribe this text

[Assistant] So if you're interested in this area of API usability, uh, we wrote a nice overview article of what we've done, uh, last summer. Uh, it's a CACM article, kind of gives a nice overview.


[User] Please transcribe this text

[Assistant] As I mentioned, we have this, uh, new paper, uh, that we just published on Monday, uh, that talks about the, uh, style guidelines and then we have lots of other


[User] Please transcribe this text

[Assistant] Work that's, uh, on our, on my site, on the Nat, Natural Programming site on API Usability, but we also are maintaining APIUsability.org, uh, which tries to collect together a lot of resources.


[User] Please transcribe this text

[Assistant] Just real quickly, I want to mention some other research projects that I didn't talk about. We're working on exploratory programming to try and help data scientists investigate things through code.


[User] Please transcribe this text

[Assistant] We're working on a conversational agent through programming by example, so it's fun to get back to programming by example.