Creating your own 1password clone
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Nieto, Carlos Martín
AgileBits, the company behind the 1password password manager, published a spec for their “opvault” format to show how confident they are in its design. This eliminates the need to reverse-engineer the encryption when trying to read from such a vault on a system where they don’t provide their tool. In this talk we’ll see an overview of the design of the format, such as the key derivation or the decision to split the meta-data from the details such as username and passwords. At the same time, the talk will follow the implementation of a library to read this format in Rust, which started as a way to practice the language but now has grown a GUI to display these entries so I can use the vault on my desktop.
 All right, so welcome to the talk about creating about one password kernel will be going through what the format that they use is and how it went around about implementing them for my own, for Fun and Profit. And they will see a bit about interesting details about what they actually did. So me and Carlos from the GitHub, I run the work and the team that runs the Git infrastructure. And for the last few years, almost anything I touch ends up having to implement some encryption. So this time I said, well, let's do something for Fun instead of just for work. And disclaimer, I'm not your crypto expert. So if you want to do, I think the steps they've taken here at one password that they're interesting, it can be useful to learn from them, but consult your own crypto and security expert, your organization, before doing stuff. So I expect most of you are going to know, but just to go a bit over what the one password is, right? So this is a password manager. What they do is they keep track of all your secrets so that you don't have to keep around like a big ledger full of your passwords and all the old passwords, and then you can't find anything. And then the alternative is, oh, let's just use the one password, which isn't good, because then, you know, Equifax gets popped, and then now your Gmail is also popped, which means everything is not popped. And really, the big advantage of a password manager is when they have a nice browser integration, because then it's easier to create a secure password and just store it and forget about it and just use that one instead of trying to come up with a clever password, which is probably not going to be as secure as what the random number generator in your kernel is going to give them. Of course, sometimes websites have, they think they're smart, so they have interesting password requirements, so you have to try a few times, and that's definitely not fun. But you know, you complain to them rather than the password managers. So this is about the one password, one password vault format called OP vault. This is the import and export format. This isn't exactly what they use in the application. They use something inside a SQLite database, but it's essentially the same. This is just the serialized version of SQLite. This is what they use for... So the app calls the backup, but essentially is what you dump out to Dropbox or iCloud or R-Sync, which is how I synced the things this morning, or whatever synchronization you have. And that will then...it will look at directory and import if any changes happen in other machines. One password itself, except for new developments, which are... We're trying to actually get your data in the cloud. They're a local password. That means all the data is just up in your file system, and then you are the one who decides exactly how you distribute these exported backups in case you do want to use multiple machines. This is all under your control. They even have a very neat local network synchronization, so you can actually synchronize with your phone without actually having to go out through the internet. It's pretty cool. So this URL where they actually document publicly their format, because the old format was secret and it wasn't as secure as it could have been. So after that, they decided, okay, well, let's just make it open. This new one that's actually secure and then let everyone know how it is, because that's how sure we are about our format. So this is a directory. They have a bunch of files. So you can see here the dot attachment ones are attachments to items. We'll see later how these all go together, essentially. You have a bunch of secrets, and then you can attach a picture or whatever is necessary. I think the export went a bit bad. So you have 16 bands. So that's just a bunch of JavaScript files with JSON in them to reduce the amount of churn that any particular file has, because this is for Dropbox to have. And Dropbox isn't necessarily updating all of the files in both machines, and then you get a weird thing, which you've probably seen if you ever tried to sync Git repositories of a Dropbox, which we keep getting comments about. So then you have a profile, which is where all the, essentially, many information lives, and then a directory of folders to organize things. So how do we then, basically, almost all of the data here is encrypted. So how do we actually get to encryption? So we have a key, right, that we need to encrypt the things, but then where do we put that key? We have plain text on the file system, because that defeats the whole point. So we can put it, but we can also encrypt that one, which only slows the problem slightly, because now we have another key that we need to store somewhere. However, this other key is something that we can create out of the user's master password. So the password you use when the program starts, you type it in, and then it unlocks everything. That password is the thing that encrypts the real encryption key that decrypts your secrets. Now your password, if you type A, that's not a secure enough key. So what we have is a function called PBKDF2, which I can never type it directly unless I actually mouth out what it means. So it stands for password-based key derivation function 2, because the one wasn't good enough. This is somewhat similar to S-crypt or B-crypt that you might know from storing user passwords in your database, except this, instead of trying to create a hash so that we can compare to see if it matches, the output of this is a key of some bytes of a particular length that's better suited for performing encryption key. This also means that, as I mentioned, similar to S-crypt or B-crypt in S-crypt, it has an assault, but it also has a number of rounds or iterations that you can configure. So the idea being that it should be slow to calculate one. So it doesn't matter too much when you type in the correct password. Maybe it takes half a second, one second to actually create the key, but it does definitely hurt someone who's trying to brute force their way into your secret vault. And we don't use the key that we get out of your password directly, because that also means that we can encrypt the real master key with this other password, and then we only have to update one particular field in this one particular file instead of trying to re-encrypt absolutely everything. It means that we don't need to re-encrypt all of the files, and then all of them just stay together, and then you can automatically increase the iteration number, which means as computer gets faster, you can ship an update that just says, well, the next update, instead of doing 50,000 iterations, you do 80,000 iterations. And then that means that you, it's no increasing the iteration, sorry, re-encrypting the data out of the password, making it more secure, is no different from doing a normal export. Yeah, sorry, I already went through this bit. So one of the things that they also do is a thing called authenticated encryption. Now this isn't always 100% relevant to an offline client, because it often requires you to have a conversation. But essentially the way that you might want to do the checksamming of data that you encrypt is you decrypt it, and then you compare the checksam of the decrypted data against some checksam that you've recovered from somewhere. This is fine except for the kind of attacks called choosing self-attacks attack, in which you can mangle the ciphertext a bit, even if you don't know what it contains. And then if you actually can establish a conversation and have the tool tell you whether it's the decrypt the text and gets mangle in a particular way, you can slowly figure out the plain text without actually ever figuring out what the key is, or without knowing the advance of the key is. Now this isn't 100% relevant for an offline tool, because it would just be, well, sorry, I can't load your vault now. But it does protect against any kind of chosen self-attacks attack. So even if someone comes up and figures out a way to stay in your machine for a while and keep twilling the bits in your vault, then they're still not going to be able to get it, because the first thing we do here is we check the checksam of the encrypted text, and then we decrypt it. This requires specific cryptographic primitives or functions. But those luckily also exist in almost any cryptographic library you will have in your system. So the one I'm using from a library is just OpenSSL, but you can also have them with common crypto, which is the one from Mac OS. And this is the one that they use in their app itself, which is primarily a Mac thing. So now we know how to create a key out of our password. That gives us a 512 key bit. Now that was split into two. One is the encryption key, and one is the Mac key. The master key is, however, actually two keys is master and overview, but we'll see that in a sec. We're using AS256, so we only need half of this key, but we still don't want to do multiple times this BBDK2. So we create, now that we have the password, now we have this other key, that with that, we decrypt the master key and the overview key. The master key is the one used for the encryption. The overview key is the one used for encrypting the metadata or the sort of high level details of items or your secrets. So for example, this lets you have the URL for the website or its title listed in your list of items, so you can look for them without actually having to decrypt the actual password. So it's less likely that someone looking at memory dump will actually find your password somewhere in your file or in your encrypted swap if you have that. So this is what the profile.js file looks like. The keys are pretty long, so I've highlighted a few. The stuff like the password hands, rounds, that kind of thing, which is sort of highlighted here. Now this does look like it's being loaded into a JavaScript engine because it's been assigned to a variable. This is necessarily the most, I mean, it looks a bit suspect that it looks like you're trying to load into some JavaScript when you're not supposed to actually trust the inputs because this is some file from somewhere. I'm not saying that this is actually vulnerable to anything, but it does look a bit weird. So you can see here the iterations are about 50,000. I looked this morning and on my current export is about 83,000, so they do continually update this data. So this profile comes from the test data that they show up in the web page, so you can actually test if you want to implement something without having to use your own vault, which, you know, which means you're less likely to actually print out all of your passwords into the terminal for debugging. So let's go back to the kinds of keys. We started with a password, we made that into two keys, and without the decrypted, these other two keys, which are actually two each, but I generally treat them as just the one because they all both go together. It's time you get the key, you split them, you split into two. The first half gives you the encryption key and the other one is the verification key. So you first use the second half to verify that the encrypted text hasn't been tampered with and then you can decrypt and then you can know that at least the right person encrypted the text, even if you don't necessarily know what's inside. And then we can now start opening all of the items. We can read those banned, you know, ABC files, and then we can load and finally we can display to the user what the list of items is. So an item is just what this thing calls each entry in the database, each secret that it's storing. Almost all of the data is encrypted. There's a few things which are like the category, what it's a favorite, the folder it contains. So this is because that way you can actually figure out if something is a favorite. So if the user says, oh, show me all of the favorites, you don't even need to bother decrypting the overview to see if it's favorite. However, the first thing the app does when you input the password is to show you all items from all vaults. So I'm not sure whether this was an idea that they had, oh, maybe some integration will actually want to do this because the app doesn't seem to be taking advantage of this. The first thing it will do is decrypt everything. All of the overviews. So the overview will consist of essentially a date when this was done, some text and some auxiliary information. Sometimes it will say, well, you created this password on this date. Or this is a great card for this bank and this person. They have a lot of somewhat specific kinds of items. You can have an outdoor license, one specific one. You have credit cards and they have a type for each credit card. But most of the time you're just going to have a generic password. Now this, each item itself also has its own keys. Not for the overview, that one is sort of less secure or less sensitive. But now that you once you have your master key for encryption, you can still decrypt this other key pair for each item. That way now you can get to the detail, which is actually the password or the credit card number or your SSN or whatever. Because the Americans also have this thing. And that means that when you, there's only very little actual content that gets encrypted under each particular key, which it doesn't necessarily protect against any known attack. But generally the less that you use any particular key, the less likely it is that you've misused it in some way. And thus you've exposed some data. So the less actual secrets that you store under each key, the harder the chance it is that even if an attack becomes possible, it's less likely that it's going to hurt you. They do however, one of the other things that the app, the actual bits provides a one password. It also just forgets your password if you wait too long without going into it or if you close your laptop and it goes to sleep, it also just forgets. It also reduces the chance that someone will manage to go and get it, but it also means that it evacuates the memory more often. Because if you were, I mean, this overview information isn't considered to be as sensitive, so it's okay if someone sees it a little bit. But it's still going to be easier to find that in a memory dump or an encrypted swap, like I mentioned, than the keys itself, which are also going to be in memory. So it's sort of trying to be secure while also not asking you for your password for every single thing. So it's kept in memory, but only some of the keys, it's trying to play around with this, trying to figure out what the recommendation is of spending time on behalf of the users just storing keys versus not. So one of the interesting things here with the, when trying to perform a checksum, so most of the time you run this authenticated encryption in which you have a chunk of data, however, because some of the data inside an item is unencrypted, like the FAV, the category. We can't run the same process, so they came up with this other process where you essentially do the same thing, but you grab all of the data and you store the HMAC, which includes all of the encrypted data plus all of the unencrypted one, except obviously for the HMAC entry in the field. Now this is a JSON object, so it's all a bunch of strings, and numbers and volumes are primarily strings. So also because there is no canonical representation for any of this, so you can't just hash the output because the multiple implementations could just produce anything. So what they do is they have the key and the value, so you have the key, you hash that and then you put the value as a string and then you hash that. However, probably because they wrote this in Objective C and that doesn't actually have Booleans, but only integers, they have a single optional Boolean in the items, which is whether it's been sent to the trash. It took me a while to figure out because I could never verify this one object until I realized that, oh, right, I'm in Rust, so I actually have Booleans, I'm going to say, print me this to strings as Boole, but because of the generic dictionary that they're using in one password implementation, it turns out that it's trying to print a one instead of a Boolean true. So that took way too long to figure out that it turns out Booleans aren't always Booleans. I don't know if this is just a side effect of them storing things in a SQL database where types aren't necessarily a real thing. But yeah, it took a while, it's not as clear as it could be, but it's the final thing I managed to read in the end. Attachments, there's not much of them. You can, each attachment specifies what item they go to. They also have an overview and a detail. So you can say, oh, this is my credit card or this is my ID, and then the actual details is just the image. Generally, very often, it would just be an image. Folders, they also have a name and a UUID. This UID is what the items refer to, to say I'm in this folder. They do have this particular kind of a folder called a smart folder, which can, for example, in the test data, for example, they say, oh, show me all of the items that contain attachments. Now this is, I assume, oh, this is probably some JavaScript or some Lua and some sandboxy thing that runs through code. But it turns out it's something in binary that they don't promise is ever going to stay stable anyway. And it turns out the base64 encoded string isn't even a properly encoded string in the test data list. I think it's like a null somewhere, or half a byte is missing or something strange. So I don't use them, so I didn't bother too much, but they're there. It's an interesting feature, but I guess you'll have to reverse engineer this binary format if you want to write something like this, like something that reads these smart folders and you're actually using them. So putting all of this data into its own library, right? So I figured, hey, I should try out something in Rust. It's the one language I don't get to use usually at work. So I figured, well, okay, I should do this since it's public, and then I've managed to live, I've mulled through by using my phone as the other one password client and then just copying it around. Because luckily most of the time, I can use my QFA key to go into pseudomode instead of having to set my password 20 times a day. So it was fine, but then I got a bit tired. And of course, the official solution from Agile Bates is, oh, you should just run Wine and run the Windows implementation from them in your Debian. It's just like, okay, cool, but no, I don't want to do that. I'm sure it works fine, but it doesn't know. Maybe that's the point. So let's try to do it, it was like a year ago, roughly. So I mentioned that we have this overview key plus the master key, and that's what you used for the items and what you used for the attachments to see all their overview versus the details. So I was going to say, okay, well, I'm a C programmer generally, so I'll just add pointers everywhere. Turns out Rust says, oh, but you have to prove to me that you're not going to free the vaults that actually contains all of this data while you're using the other thing. And then I'm like, no, but it's fine. I've been writing C for a decade. I rarely suck fault. But so in the end, I sort of took the code's way out and added a reference counter around it. It was fine, but I still want to go back in and actually fix it more now that the compiler has better error messages, so it's not shouting at me as much. And also we'll shout out to CERTI, the CERTI session library. It's very versatile. It lets me specify almost completely what I do in code and sort of statically in compile time and it will generate all of the code I want. It's pretty cool. So once I, now we know how to decode this, the actually encrypted data, and now we have a bunch of JSON blobs that aren't actually documented because that's application data. But at least JSON is self-documenting or something. So at least we can actually see what the keys and values are and just build up a bunch of abstracts and just do it, just have the CERTI session library do it for me. However, there are some things that aren't quite clear whether it's just the way the app, someone wrote it and then forgot about it or whatever. But there's, when you have a login or something you submitted through a web form and it gets stored from your browser, from the integration, right? They have a type for a field that's password. But then I noticed that half of my passwords were just printed in plain text in my UI that I was writing. I was like, well, what's going on here? Turns out a password is a password when it has a password type or when it's a normal text type and its name or its designation are also password. So the app knows it's a password. It doesn't store it as a password. I'm not sure why that's it, right? So this is a bit of the code I have there where I'm saying, okay, well, is this a password or does it look like a password? This is what you call duck typing or something. There was also an interesting, so when you have the more specific ones, we have outdoor license, whatever. It actually has some very specific fields. Just with one string, but they all have different kinds. I started doing it within each half of their own struct because they want to be proper and have types and safety and whatever, right? But then it turns out that they all are the same and two that have a different. So at the end of the day, this is where my second trial to cert that goes because so here we can see K and so this is the key, the name, the value, and then some attributes. The value can be a number, a string, or this whole other struct which is address. This is all optional and certainly let me just say, hey, there's this value field. It can be one of these three things. Just try them all and it will just write the code for me that tries to figure, hey, is this a number? Oh, this matches the number. Is this a string? Okay, or is this another object that has all of these fields? Oh, then I'm going to pack it into this address field which is another very specific thing that they have where otherwise it's mostly strings. And on this sort of specific versus not specificity that they have in a few places, they seem to have this form of get text by default. So, you see, oh, the name or the designation for this field is password. Now for a bunch of these things, they have a designation which... So they have a name which is the field name in the browser. But they also have a thing, for example, expredate game, whatever. So game is for the outdoor license because apparently they have one very specific field for what kind of animal to allow to kill. They have... None of these are actually... Just one slide. None of these just have the English or human name of this. They have something that presumably you can get text, which is actually cool because most of the time you're stuck with the English definitions of things. Whereas here, they default to, well, this is just something and then we're going to update the app as things happen. So this is the source code. The first one is the library. The second one is the GUI. You shouldn't use the GUI right now because for debugging purposes, it will actually print everything into the command line. Into where you're running it. So this is going to print out of your passwords. So don't use that without commenting that out. Thanks. Thank you very much.
 All right, so welcome to the talk about creating about one password kernel will be going through  what the format that they use is and how it went around about implementing them for my  own, for Fun and Profit.  And they will see a bit about interesting details about what they actually did.  So me and Carlos from the GitHub, I run the work and the team that runs the Git infrastructure.  And for the last few years, almost anything I touch ends up having to implement some encryption.  So this time I said, well, let's do something for Fun instead of just for work.  And disclaimer, I'm not your crypto expert.  So if you want to do, I think the steps they've taken here at one password that they're interesting,  it can be useful to learn from them, but consult your own crypto and security expert, your organization,  before doing stuff.  So I expect most of you are going to know, but just to go a bit over what the one password  is, right?  So this is a password manager.  What they do is they keep track of all your secrets so that you don't have to keep around  like a big ledger full of your passwords and all the old passwords, and then you can't  find anything.  And then the alternative is, oh, let's just use the one password, which isn't good, because  then, you know, Equifax gets popped, and then now your Gmail is also popped, which means  everything is not popped.  And really, the big advantage of a password manager is when they have a nice browser integration,  because then it's easier to create a secure password and just store it and forget about  it and just use that one instead of trying to come up with a clever password, which is  probably not going to be as secure as what the random number generator in your kernel  is going to give them.  Of course, sometimes websites have, they think they're smart, so they have interesting  password requirements, so you have to try a few times, and that's definitely not fun.  But you know, you complain to them rather than the password managers.  So this is about the one password, one password vault format called OP vault.  This is the import and export format.  This isn't exactly what they use in the application.  They use something inside a SQLite database, but it's essentially the same.  This is just the serialized version of SQLite.  This is what they use for...  So the app calls the backup, but essentially is what you dump out to Dropbox or iCloud or  R-Sync, which is how I synced the things this morning, or whatever synchronization you have.  And that will then...it will look at directory and import if any changes happen in other  machines.  One password itself, except for new developments, which are...  We're trying to actually get your data in the cloud.  They're a local password.  That means all the data is just up in your file system, and then you are the one who  decides exactly how you distribute these exported backups in case you do want to use multiple  machines.  This is all under your control.  They even have a very neat local network synchronization, so you can actually synchronize with your  phone without actually having to go out through the internet.  It's pretty cool.  So this URL where they actually document publicly their format, because the old format was secret  and it wasn't as secure as it could have been.  So after that, they decided, okay, well, let's just make it open.  This new one that's actually secure and then let everyone know how it is, because that's  how sure we are about our format.  So this is a directory.  They have a bunch of files.  So you can see here the dot attachment ones are attachments to items.  We'll see later how these all go together, essentially.  You have a bunch of secrets, and then you can attach a picture or whatever is necessary.  I think the export went a bit bad.  So you have 16 bands.  So that's just a bunch of JavaScript files with JSON in them to reduce the amount of  churn that any particular file has, because this is for Dropbox to have.  And Dropbox isn't necessarily updating all of the files in both machines, and then you  get a weird thing, which you've probably seen if you ever tried to sync Git repositories  of a Dropbox, which we keep getting comments about.  So then you have a profile, which is where all the, essentially, many information lives,  and then a directory of folders to organize things.  So how do we then, basically, almost all of the data here is encrypted.  So how do we actually get to encryption?  So we have a key, right, that we need to encrypt the things, but then where do we put that  key?  We have plain text on the file system, because that defeats the whole point.  So we can put it, but we can also encrypt that one, which only slows the problem slightly,  because now we have another key that we need to store somewhere.  However, this other key is something that we can create out of the user's master password.  So the password you use when the program starts, you type it in, and then it unlocks everything.  That password is the thing that encrypts the real encryption key that decrypts your secrets.  Now your password, if you type A, that's not a secure enough key.  So what we have is a function called PBKDF2, which I can never type it directly unless  I actually mouth out what it means.  So it stands for password-based key derivation function 2, because the one wasn't good enough.  This is somewhat similar to S-crypt or B-crypt that you might know from storing user passwords  in your database, except this, instead of trying to create a hash so that we can compare  to see if it matches, the output of this is a key of some bytes of a particular length  that's better suited for performing encryption key.  This also means that, as I mentioned, similar to S-crypt or B-crypt in S-crypt, it has an  assault, but it also has a number of rounds or iterations that you can configure.  So the idea being that it should be slow to calculate one.  So it doesn't matter too much when you type in the correct password.  Maybe it takes half a second, one second to actually create the key, but it does definitely  hurt someone who's trying to brute force their way into your secret vault.  And we don't use the key that we get out of your password directly, because that also  means that we can encrypt the real master key with this other password, and then we  only have to update one particular field in this one particular file instead of trying  to re-encrypt absolutely everything.  It means that we don't need to re-encrypt all of the files, and then all of them just  stay together, and then you can automatically increase the iteration number, which means  as computer gets faster, you can ship an update that just says, well, the next update, instead  of doing 50,000 iterations, you do 80,000 iterations.  And then that means that you, it's no increasing the iteration, sorry, re-encrypting the data  out of the password, making it more secure, is no different from doing a normal export.  Yeah, sorry, I already went through this bit.  So one of the things that they also do is a thing called authenticated encryption.  Now this isn't always 100% relevant to an offline client, because it often requires  you to have a conversation.  But essentially the way that you might want to do the checksamming of data that you encrypt  is you decrypt it, and then you compare the checksam of the decrypted data against some  checksam that you've recovered from somewhere.  This is fine except for the kind of attacks called choosing self-attacks attack, in which  you can mangle the ciphertext a bit, even if you don't know what it contains.  And then if you actually can establish a conversation and have the tool tell you whether it's the  decrypt the text and gets mangle in a particular way, you can slowly figure out the plain text  without actually ever figuring out what the key is, or without knowing the advance of  the key is.  Now this isn't 100% relevant for an offline tool, because it would just be, well, sorry,  I can't load your vault now.  But it does protect against any kind of chosen self-attacks attack.  So even if someone comes up and figures out a way to stay in your machine for a while  and keep twilling the bits in your vault, then they're still not going to be able to  get it, because the first thing we do here is we check the checksam of the encrypted text,  and then we decrypt it.  This requires specific cryptographic primitives or functions.  But those luckily also exist in almost any cryptographic library you will have in your  system.  So the one I'm using from a library is just OpenSSL, but you can also have them with common  crypto, which is the one from Mac OS.  And this is the one that they use in their app itself, which is primarily a Mac thing.  So now we know how to create a key out of our password.  That gives us a 512 key bit.  Now that was split into two.  One is the encryption key, and one is the Mac key.  The master key is, however, actually two keys is master and overview, but we'll see that  in a sec.  We're using AS256, so we only need half of this key, but we still don't want to do multiple  times this BBDK2.  So we create, now that we have the password, now we have this other key, that with that,  we decrypt the master key and the overview key.  The master key is the one used for the encryption.  The overview key is the one used for encrypting the metadata or the sort of high level details  of items or your secrets.  So for example, this lets you have the URL for the website or its title listed in your  list of items, so you can look for them without actually having to decrypt the actual password.  So it's less likely that someone looking at memory dump will actually find your password  somewhere in your file or in your encrypted swap if you have that.  So this is what the profile.js file looks like.  The keys are pretty long, so I've highlighted a few.  The stuff like the password hands, rounds, that kind of thing, which is sort of highlighted  here.  Now this does look like it's being loaded into a JavaScript engine because it's been assigned  to a variable.  This is necessarily the most, I mean, it looks a bit suspect that it looks like you're trying  to load into some JavaScript when you're not supposed to actually trust the inputs because  this is some file from somewhere.  I'm not saying that this is actually vulnerable to anything, but it does look a bit weird.  So you can see here the iterations are about 50,000.  I looked this morning and on my current export is about 83,000, so they do continually update  this data.  So this profile comes from the test data that they show up in the web page, so you can actually  test if you want to implement something without having to use your own vault, which, you know,  which means you're less likely to actually print out all of your passwords into the terminal  for debugging.  So let's go back to the kinds of keys.  We started with a password, we made that into two keys, and without the decrypted, these  other two keys, which are actually two each, but I generally treat them as just the one  because they all both go together.  It's time you get the key, you split them, you split into two.  The first half gives you the encryption key and the other one is the verification key.  So you first use the second half to verify that the encrypted text hasn't been tampered  with and then you can decrypt and then you can know that at least the right person encrypted  the text, even if you don't necessarily know what's inside.  And then we can now start opening all of the items.  We can read those banned, you know, ABC files, and then we can load and finally we can display  to the user what the list of items is.  So an item is just what this thing calls each entry in the database, each secret that it's  storing.  Almost all of the data is encrypted.  There's a few things which are like the category, what it's a favorite, the folder it contains.  So this is because that way you can actually figure out if something is a favorite.  So if the user says, oh, show me all of the favorites, you don't even need to bother decrypting  the overview to see if it's favorite.  However, the first thing the app does when you input the password is to show you all  items from all vaults.  So I'm not sure whether this was an idea that they had, oh, maybe some integration will  actually want to do this because the app doesn't seem to be taking advantage of this.  The first thing it will do is decrypt everything.  All of the overviews.  So the overview will consist of essentially a date when this was done, some text and some  auxiliary information.  Sometimes it will say, well, you created this password on this date.  Or this is a great card for this bank and this person.  They have a lot of somewhat specific kinds of items.  You can have an outdoor license, one specific one.  You have credit cards and they have a type for each credit card.  But most of the time you're just going to have a generic password.  Now this, each item itself also has its own keys.  Not for the overview, that one is sort of less secure or less sensitive.  But now that you once you have your master key for encryption, you can still decrypt  this other key pair for each item.  That way now you can get to the detail, which is actually the password or the credit card  number or your SSN or whatever.  Because the Americans also have this thing.  And that means that when you, there's only very little actual content that gets encrypted  under each particular key, which it doesn't necessarily protect against any known attack.  But generally the less that you use any particular key, the less likely it is that you've misused  it in some way.  And thus you've exposed some data.  So the less actual secrets that you store under each key, the harder the chance it is  that even if an attack becomes possible, it's less likely that it's going to hurt you.  They do however, one of the other things that the app, the actual bits provides a one password.  It also just forgets your password if you wait too long without going into it or if  you close your laptop and it goes to sleep, it also just forgets.  It also reduces the chance that someone will manage to go and get it, but it also means  that it evacuates the memory more often.  Because if you were, I mean, this overview information isn't considered to be as sensitive,  so it's okay if someone sees it a little bit.  But it's still going to be easier to find that in a memory dump or an encrypted swap,  like I mentioned, than the keys itself, which are also going to be in memory.  So it's sort of trying to be secure while also not asking you for your password for  every single thing.  So it's kept in memory, but only some of the keys, it's trying to play around with this,  trying to figure out what the recommendation is of spending time on behalf of the users  just storing keys versus not.  So one of the interesting things here with the, when trying to perform a checksum, so  most of the time you run this authenticated encryption in which you have a chunk of data,  however, because some of the data inside an item is unencrypted, like the FAV, the category.  We can't run the same process, so they came up with this other process where you essentially  do the same thing, but you grab all of the data and you store the HMAC, which includes  all of the encrypted data plus all of the unencrypted one, except obviously for the  HMAC entry in the field.  Now this is a JSON object, so it's all a bunch of strings, and numbers and volumes  are primarily strings.  So also because there is no canonical representation for any of this, so you can't just hash the  output because the multiple implementations could just produce anything.  So what they do is they have the key and the value, so you have the key, you hash that  and then you put the value as a string and then you hash that.  However, probably because they wrote this in Objective C and that doesn't actually have  Booleans, but only integers, they have a single optional Boolean in the items, which is whether  it's been sent to the trash.  It took me a while to figure out because I could never verify this one object until  I realized that, oh, right, I'm in Rust, so I actually have Booleans, I'm going to say,  print me this to strings as Boole, but because of the generic dictionary that they're using  in one password implementation, it turns out that it's trying to print a one instead of  a Boolean true.  So that took way too long to figure out that it turns out Booleans aren't always Booleans.  I don't know if this is just a side effect of them storing things in a SQL database  where types aren't necessarily a real thing.  But yeah, it took a while, it's not as clear as it could be, but it's the final  thing I managed to read in the end.  Attachments, there's not much of them.  You can, each attachment specifies what item they go to.  They also have an overview and a detail.  So you can say, oh, this is my credit card or this is my ID, and then the actual details  is just the image.  Generally, very often, it would just be an image.  Folders, they also have a name and a UUID.  This UID is what the items refer to, to say I'm in this folder.  They do have this particular kind of a folder called a smart folder, which can, for example,  in the test data, for example, they say, oh, show me all of the items that contain attachments.  Now this is, I assume, oh, this is probably some JavaScript or some Lua and some sandboxy  thing that runs through code.  But it turns out it's something in binary that they don't promise is ever going to stay  stable anyway.  And it turns out the base64 encoded string isn't even a properly encoded string in the  test data list.  I think it's like a null somewhere, or half a byte is missing or something strange.  So I don't use them, so I didn't bother too much, but they're there.  It's an interesting feature, but I guess you'll have to reverse engineer this binary format  if you want to write something like this, like something that reads these smart folders  and you're actually using them.  So putting all of this data into its own library, right?  So I figured, hey, I should try out something in Rust.  It's the one language I don't get to use usually at work.  So I figured, well, okay, I should do this since it's public, and then I've managed to  live, I've mulled through by using my phone as the other one password client and then  just copying it around.  Because luckily most of the time, I can use my QFA key to go into pseudomode instead of  having to set my password 20 times a day.  So it was fine, but then I got a bit tired.  And of course, the official solution from Agile Bates is, oh, you should just run Wine  and run the Windows implementation from them in your Debian.  It's just like, okay, cool, but no, I don't want to do that.  I'm sure it works fine, but it doesn't know.  Maybe that's the point.  So let's try to do it, it was like a year ago, roughly.  So I mentioned that we have this overview key plus the master key, and that's what you  used for the items and what you used for the attachments to see all their overview versus  the details.  So I was going to say, okay, well, I'm a C programmer generally, so I'll just add pointers  everywhere.  Turns out Rust says, oh, but you have to prove to me that you're not going to free the vaults  that actually contains all of this data while you're using the other thing.  And then I'm like, no, but it's fine.  I've been writing C for a decade.  I rarely suck fault.  But so in the end, I sort of took the code's way out and added a reference counter around  it.  It was fine, but I still want to go back in and actually fix it more now that the compiler  has better error messages, so it's not shouting at me as much.  And also we'll shout out to CERTI, the CERTI session library.  It's very versatile.  It lets me specify almost completely what I do in code and sort of statically in compile  time and it will generate all of the code I want.  It's pretty cool.  So once I, now we know how to decode this, the actually encrypted data, and now we have  a bunch of JSON blobs that aren't actually documented because that's application data.  But at least JSON is self-documenting or something.  So at least we can actually see what the keys and values are and just build up a bunch of  abstracts and just do it, just have the CERTI session library do it for me.  However, there are some things that aren't quite clear whether it's just the way the  app, someone wrote it and then forgot about it or whatever.  But there's, when you have a login or something you submitted through a web form and it gets  stored from your browser, from the integration, right?  They have a type for a field that's password.  But then I noticed that half of my passwords were just printed in plain text in my UI that  I was writing.  I was like, well, what's going on here?  Turns out a password is a password when it has a password type or when it's a normal  text type and its name or its designation are also password.  So the app knows it's a password.  It doesn't store it as a password.  I'm not sure why that's it, right?  So this is a bit of the code I have there where I'm saying, okay, well, is this a password  or does it look like a password?  This is what you call duck typing or something.  There was also an interesting, so when you have the more specific ones, we have outdoor  license, whatever.  It actually has some very specific fields.  Just with one string, but they all have different kinds.  I started doing it within each half of their own struct because they want to be proper  and have types and safety and whatever, right?  But then it turns out that they all are the same and two that have a different.  So at the end of the day, this is where my second trial to cert that goes because so  here we can see K and so this is the key, the name, the value, and then some attributes.  The value can be a number, a string, or this whole other struct which is address.  This is all optional and certainly let me just say, hey, there's this value field.  It can be one of these three things.  Just try them all and it will just write the code for me that tries to figure, hey, is  this a number?  Oh, this matches the number.  Is this a string?  Okay, or is this another object that has all of these fields?  Oh, then I'm going to pack it into this address field which is another very specific thing  that they have where otherwise it's mostly strings.  And on this sort of specific versus not specificity that they have in a few places, they seem  to have this form of get text by default.  So, you see, oh, the name or the designation for this field is password.  Now for a bunch of these things, they have a designation which...  So they have a name which is the field name in the browser.  But they also have a thing, for example, expredate game, whatever.  So game is for the outdoor license because apparently they have one very specific field  for what kind of animal to allow to kill.  They have...  None of these are actually...  Just one slide.  None of these just have the English or human name of this.  They have something that presumably you can get text, which is actually cool because most  of the time you're stuck with the English definitions of things.  Whereas here, they default to, well, this is just something and then we're going to  update the app as things happen.  So this is the source code.  The first one is the library.  The second one is the GUI.  You shouldn't use the GUI right now because for debugging purposes, it will actually  print everything into the command line.  Into where you're running it.  So this is going to print out of your passwords.  So don't use that without commenting that out.  Thanks.  Thank you very much.
2017_Creating_your_own_1password_clone_UJl4GJSZD7M_0_59.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_60_155.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_156_227.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_228_275.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_276_407.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_408_659.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_660_1559.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_1560_3275.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_3276_5723.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_5724_7199.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_7200_11987.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_11988_12239.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_12240_15179.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_15180_17579.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_17580_19487.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_19488_20831.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_20832_26915.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_26916_30287.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_30288_30839.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_30840_32603.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_32604_36047.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_36048_36551.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_36552_38195.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_38196_38555.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_38556_40571.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_40572_41951.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_41952_42539.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_42540_42719.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_42720_42791.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_42792_42839.png 2017_Creating_your_own_1password_clone_UJl4GJSZD7M_42840_42870.png