Using Password-based Encryption on Android
Why password-based encryption is needed KeyChain class. However, as discussed here, github).
How not to generate a key from a password: padded password
int keyLength = 128; byte[] keyBytes = new byte[keyLength / 8]; // explicitly fill with zeros Arrays.fill(keyBytes, (byte) 0x0); // if password is shorter then key length, it will be zero-padded // to key length byte[] passwordBytes = password.getBytes("UTF-8"); int length = passwordeBytes.length < keyBytes.length ? passwordBytes.length : keyBytes.length; System.arraycopy(passwordBytes, 0, keyBytes, 0, length); SecretKey key = new SecretKeySpec(keyBytes, "AES");).
- if the password is shorter than the key size, the fixed padding further reduces the key space. For example, if the user picks up an 8-character password, that would result in roughly 2^52 possible keys. That is less even than DES's 56 bit key which has been considered weak for ages and can be brute-forced in less than a day using commercial hardware.
-.
You might think that no one would use such a naive key derivation scheme, but as it turns out, even fairly popular key manager apps are known to have used it.
To sum this up: a symmetric encryption key needs to be random to provide sufficient security, and user-entered passwords are a poor source of randomness. Don't use them as is to construct a key.
How not to generate a key from a password: SHA1PRNG
How not to generate a key from a password: SHA1PR StackOverflow:
KeyGenerator kgen = KeyGenerator.getInstance("AES"); SecureRandom sr = SecureRandom.getInstance("SHA1PRNG"); byte[] seed = password.getBytes("UTF-8"); sr.setSeed(seed); kgen.init(KEY_LENGTH, sr); SecretKey key = kgen.generateKey();
This creates a random generator instance (
SecureRandom) using the
SHA1PRNGPRNG algorithm (which is currently the only RNG algorithm available on commercial Android devices), and seeds it with the password bytes. A
KeyGeneratoris then initialized with the
SecureRandominstance, making sure that our password-seeded PRNG will be used when generating keys. Lastly, since a
KeyGeneratorfor a symmetric algorithm simply requests a number of bits equal to the key size from the underlying (or system) RNG, we get a pseudorandom secret key based on the used password. JavaDoc entry for Android's
SecureRandomsays so: 'Not guaranteed to be compatible with the SHA1PRNG algorithm on the reference implementation.' The code above when run on Android and on a desktop system using Java SE produces the following 128 bit keys from the password string 'password'. Note that those may differ even between different Android platform or Java SE versions:
Android: 80A4495EF27725345AB3AFA08CE3A692 Java SE: 2470C0C06DEE42FD1618BB99005ADCA2
In short: while this method is slightly better than the previous one, it doesn't effectively prevent from brute force attacks and is not portable. Don't use it. Update: As of Android 4.2, the default SHA1PRNG provider is based on OpenSSL and this method doesn't work out of the box. If you need to use it for compatibility reasons, you have to explicitly specify the
"Crypto"provider when getting a
SecureRandominstance. But again, don't use it.
Proper key derivation: PKCS#5 and PKCS#12
A standard way to derive a symmetric encryption key from a password is defined in PKCS#5 (Public Key Cryptography Standard) published by RSA (the company). It was originally developed for generating DES keys, but the current versions (2.0 and draft of 2.1) extend it to be algorithm independent. Version 2.0 is also published as RFC 2898.
The standard is based on two main ideas: using a salt to protect from table-assisted (pre-computed) dictionary attacks (salting) and using a large iteration count to make the key derivation computationally expensive (key stretching)..
PKCS#5 defines two key derivation functions, aptly named CBC mode. However, since this is not enough for algorithms with longer keys such as 3DES and AES, PBKDF1 shouldn't be used and is only left in the standard for backward compatibility reasons. HMAC-SHA1 being the most common. The password is used as the HMAC key and the salt takes the role of the message. Unlike PBKDF1, PBKDF2 doesn't specify how to derive an IV (initialization vector), so a randomly generated one is used.
Android's main JCE provider (Bouncy Castle) currently only supports
Android's main JCE provider (Bouncy Castle) currently only supports
PBKDF2WithHmacSHA1. Let's see how to use it to encrypt data with a 256 bit AES key derived from a password:
String password = "password"; int iterationCount = 1000; int keyLength = 256; int saltLength = keyLength / 8; // same size as key output SecureRandom random = new SecureRandom(); byte[] salt = new byte[saltLength]; randomb.nextBytes(salt);"); byte[] iv = new byte[cipher.getBlockSize()); random.nextBytes(iv); IvParameterSpec ivParams = new IvParameterSpec(iv); cipher.init(Cipher.ENCRYPT_MODE, key, ivParams); byte[] ciphertext = cipher.doFinal(plaintext.getBytes("UTF-8"));
Here we generate a random salt and use 1000 iterations to initialize the
SecretKeyFactorywhich generates our key. The last step of key generation might be a little confusing though: we don't use the
SecretKeyproduced by the factory as is, but use its encoded value to create a new
SecretKeySpecobject. That is done because the output of
generateSecret()is actually a
PBEKeyinstance which does not contain an initialized IV -- the
Cipherobject expects that from a
PBEKeyand will throw an exception if it is not present. The iteration count is as recommended by PKCS#5, but that standard was written a while ago, so you might want to increase it. For some perspective, AES 256 bit keys used to encrypt backups in Android 4.0 (ICS) are derived using 10,000 iterations and a 512 bit salt; iOS 4.0 also uses 10,000 iterations. The size of the salt should typically match the key size, for example 16 bytes when using a AES with a 128 bit key (128 / 8 = 16). Next we generate a random IV, initialize the cipher and output the cipher text.
To be able to decrypt the cipher text we need: the password, the iteration count, the salt and the IV. The password will be input by the user, and the iteration count is generally fixed (if you decide to make it variable, you need to store it along with the other parameters), so that leaves the salt and the IV. As discussed above, the salt is not a secret, and neither is the IV. Thus they can be saved along with the cipher text. If they are stored in a single blob/file, some sort of structure is needed to be able the parse it into its components. The sample app 'saves' the encrypted message to a Base64-encoded string and simply concatenates the salt, IV and cipher text delimited by "]" (any character not used Base64 will do). Decryption is very similar to the code above, except that the salt and IV are not generated randomly, but retrieved from the encrypted message.
String[] fields = ciphertext.split("]"); byte[] salt = fromBase64(fields[0]); byte[] iv = fromBase64(fields[1]); byte[] cipherBytes = fromBase64(fields[2]); // as above SecretKey key = deriveKeyPbkdf2(salt, password); Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding"); IvParameterSpec ivParams = new IvParameterSpec(iv); cipher.init(Cipher.DECRYPT_MODE, key, ivParams); byte[] plaintext = cipher.doFinal(cipherBytes); String plainrStr = new String(plaintext , "UTF-8");
Another standard key derivation mechanism is the one defined in PKCS#12.
PBEWITHSHA256AND256BITAES-CBC-BC..
Derivation speed.
Conclusion
PBKDF2WithHmacSHA1. In short: when deriving a key from a password use
PBKDF2WithHmacSHA1, a sufficiently long randomly generated salt and an iteration count suitable for your app.
You mention that using the keychain API in ICS would not be a good fit for key management because it's only for storing certificates and their private key. But is it technically an option to use such a private key, either directly or through further derivations, to encrypt application data?
Also, and this is probably a beginners question, but I'm unclear on how the private key is made available to an application if there's a password required to access that key. And is the keychain insecure on a rooted phone or through bootloader hacking etc?
Technically you could. Once you get a PrivateKey reference you can perform any cryptographic operation with it. However, it might not be a good fit because: 1) size of encrypted data is limited by key size; 2) anything you encrypt with the private key can be decrypted with the public key, so you'll have to keep that secret as well. The current interface is tailored for usage with signing and client authentication and not really encryption. You can use the lower interface to store secret keys directly though, although this can break in future versions (see the 'Storing application secrets' article).
As for protecting the key, it is encrypted only on disk. Once you unlock the phone the decryption key is stored in memory and keys are decrypted on demand, when you access them. It is 'secure' against rooting in the sense that having root access doesn't let you access the keys if you don't know the keystore unlock password. Additionally, on Jelly Bean (4.1) Android can take advantage of hardware to protect keys.
Read the trust/credential storage implementation articles if you are interested in the details.
HTH
Good point regarding the public key. I would use the private key as if it's a user-supplied password such that it would get used to derive the actual key for encryption operations. So I'm thinking that I would not need to protect the public key in that case and I could use it (along with its certificate and private key) for mutual authentication purposes, if needed.
I guess I'm still unclear on the inherent security of a keystore. If the private key is protected with a password, then my application would need to supply the password directly, right? keyStore.getKey(String alias, char[] password). But with ICS, the new KeyChain API (I read your post on the ICS trust store implementation :-) doesn't seem to require the password to obtain the private key, so I guess any application can use any identity in the store?
Following a factory reset, I cannot access any more to the crypted data on my microSD.
I had selected the option "crypt only used memory", and it looks like the crypted data are still there (500 Mbytes are missing on my microSD).
If I go on Linux and manage to access to this crypted partition, will it be possible to get back the crypted data? I know the password I used to crypt the data...but I do not know what kind of software I could use to enter my password....
Thanks a lot if you have any response to this.
florian
I found this blog to be very much helpful. Thanks for explaining various options for using PBE.
But I do have very basic Question. For generating keys, we use user entered password, but my question is , where do we store this password for android app& how do we validate the user input?
where do we store the password(on the device?) & how is it guaranteed that its stored securely?
Thanks,
Rahul
For other options regarding protecting keys, see this article:
I guess Iam missing something!!
The point I am missing is, where do we store the user entered password? If we can store the password off the device & retrieve it for the validation, can't we do the same thing with the keys? then why do we need PBE?
Thanks,
Rahul
What are you trying to do?
I am just trying to encrypt the sensitive data of the user(stored on device).
So the app prompts the password from the user(setup by user 1st time), & with that password , we generate the key everytime & use the same for decrypting the user data & encrypting the data while storing.
But, to generate the key, we need to validate the password entered by user.(if its same as the setup one). so how do we validate that password?
If that password is supposed to be stored off the device & retreieved for the validation, why can't we store the keys themeselves off the device & retrieve them to encrypt & decrypt ?
Thanks,
The password is *not* supposed to be stored anywhere, but entered by the user. You cannot realistically expect a user to remember and enter a random 256-bit key, that is why you use a password instead. You might want to re-read the article, all of this is covered in the first half.
Now I got it. Thanks again !!
but 1 quick question, that 'known value' is some hard-coded value in application.
isn't this a concern? I guess its still safe as it's just used to comparison.
Thanks
When I enter the invalid password & try to decrypt the encrypted known value, Iam getting below exception.
10-11 11:01:28.836: E/AndroidRuntime(527): Caused by: javax.crypto.BadPaddingException: pad block corrupted
10-11 11:01:28.836: E/AndroidRuntime(527): at org.bouncycastle.jce.provider.JCEBlockCipher.engineDoFinal(JCEBlockCipher.java:715)
Can we safely assume that we get this exception if we try to decrypt the encrypted known string with the key generated from wrong password?
If I enter correct password, its working fine.
any thoughts?
BTW, Iam using PBKDF2_DERIVATION_ALGORITHM & is not supported in api level 8. any suggestions?
You article provided me the right insight of encrypting and decrypting using the same password. I implemented the way you had mentioned in your article.
the Code although works, but when i continuously running throws "Given final block not properly padded" some times and others as well. I feel that i making a small error some where.
Can you help me to resolve the issue
import java.security.SecureRandom;
import java.security.spec.KeySpec; MCrypt {
private int iterationCount = 10000;
private int saltLength = 8; // bytes; 64 bits
private int keyLength = 128;
public static void main(String[] args) throws Exception {
MCrypt mc = new MCrypt();
String encryptedData = mc.encrypt("1234");
MCrypt mc1 = new MCrypt();
System.out.println(new String(mc1.decrypt(new String(encryptedData),
"1234"), "UTF-8"));
}
public MCrypt() {
}
public String encrypt(String text) throws Exception {
if (text == null || text.length() == 0)
throw new Exception("Empty string");
byte[] encrypted = null;
SecureRandom random = new SecureRandom();
byte[] salt = new byte[saltLength];
random.nextBytes(salt);
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
byte[] iv = new byte[cipher.getBlockSize()];
random.nextBytes(iv);
IvParameterSpec ivParams = new IvParameterSpec(iv);
KeySpec keySpec = new PBEKeySpec(text.init(Cipher.ENCRYPT_MODE, key, ivParams);
encrypted = cipher.doFinal(text.getBytes("UTF-8"));
StringBuffer strBuf = new StringBuffer();
strBuf.append(new String(encrypted));
strBuf.append("]");
strBuf.append(new String(salt));
strBuf.append("]");
strBuf.append(new String(iv));
return new String(Base64.encodeBytes(strBuf.toString().getBytes()));
}
public byte[] decrypt(String code, String pwd) throws Exception {
if (code == null || code.length() == 0)
throw new Exception("Empty string");
String[] fields = new String(Base64.decode(code)).split("]");
byte[] cipherBytes = fields[0].getBytes();
byte[] salt = fields[1].getBytes();
byte[] iv = fields[2].getBytes();
KeySpec keySpec = new PBEKeySpec(pwd");
IvParameterSpec ivParams = new IvParameterSpec(iv);
byte[] decrypted = null;
cipher.init(Cipher.DECRYPT_MODE, key, ivParams);
decrypted = cipher.doFinal(cipherBytes);
return decrypted;
}
}
I will try to post in stackoverflow. Anyways thanks for the suggestion and the post.
Correct me if I'm wrong but I seem to understand that it's necessary condition that the password, used to derivate the key, is entered by the user and not stored in source code. But what if the users forgets his password?
I actually wrote code for an earlier version of Android that used this (even though you don't recommend using it):
SecureRandom.getInstance("SHA1PRNG");
When my app ran on a later version of Android, the decryption failed. From StackOverflow, I found out that Google had altered something in the latest release and I was required to use this instead:
SecureRandom.getInstance("SHA1PRNG", "Crypto");
Google essentially destroy backward compatibility when they did this and now I seriously question why anyone would want to use Android's built-in encryption. Makes no sense that your app encrypts it on one version of Android but breaks on another. And if they broke compatibility this time, what's to stop them from repeating this again?
Would it not be better to download the Bouncy Castle jar files and install them with your app and rely upon that instead? Then you wouldn't have to worry about Google breaking compatibility.
Another question. It seems that you are trying to stress the importance of generating a random key and not relying upon a user's password. Would just UUID to generate a random 128 bit number be the simplest and best solution? I use that function throughout my code.
As for the JCE, etc. implementation on Android, yes it does use Bouncy Castle, but that is an implementation detail you cannot rely on. For example, in recent version a lot of the default algorithm implementations have been moved to native code (OpenSSL). If you really need to guarantee that your code works with the same JCE implementation across different Android versions, bundling Spongy Castle (a repackaged version of Bouncy Castle) is an option.
The UUID is random, and therefore not reproducible. It doesn't really work for key derivation. If you need a random key, it might work, but then you can simply (and had better) use SecureRandom.
As for Spongy Castle, etc. simply use ProGuard it will strip unused code. You are very unlikely to use the whole of SC, so most of it will be removed.
Thanks.
My real problem here was to figure out that the password in your sample-code is a passphrase and the plaintext is to be meant the password2be2encrypted. And that the passphrase resp. pin remains constant while the password change. It took my quite a while to figure out the distinction of the mixture of the term password.
Suggestion: password ==> passphraseOrPin and plaintext ==> password2be2encrypted
my 2 cents
Also I think 'randomb' should be changed to 'random' in one of them (CTRL-F it).
I am currently working on the encryption program that encrypts XML strings which are stored in the files using AES. This post is also related to the AES encryption. However, the problem is that you generate an encryption key using password while I need to generate a secret key randomly. The idea is that I want to wrap symmetric key with public/private keys and store them in the KeyStore which became a public API from Jelly bean 4.3 (I have read this post as well). So I am not sure how to properly generate a secret key.
I am currently generating it like this:;
}
And then using secretKeySpec to initialize cipher.
byte[] convertedKeyToByte = Key.getEncoded();
SecretKeySpec skeySpec = new SecretKeySpec(convertedKeyToByte, 0, convertedKeyToByte.length,
AESalgorithm);
Is it an inefficient way of generating and applying key?
Yours sincerely,
Sergejs
Check the JCE specs for details, but you don't really need the whole SecretKeySpect part.
Thank you very much for your reply. I am sorry, I will not post unrelated questions anymore.
Kind Regards,
Sergejs
Have you used DexGuard at all? What did you think?
Brian
What should be the "password" value for sms-based authentication apps (like Whats app)? Or for apps that use OAuth, where the token is generated on the server and needs to be stored securely on the device?
In such apps, the server sends a hash via API response and a code via sms, The user enters this code and this is sent to the server along with the original hash. If the values are equal, the user is authenticated.
Can the concat of the hash + code be used as the "password" here?
Also, since this generated key be used as an API key to distinguish one user from another, since the client and server can generate the same value?
-Thanx
facing similar scenario.. How to implement forgot password module for such an application. the most stringent way... ?
A very good post with a lot of details. Thank you!
Where can I find the sample app?
Thanks!
I have tested your android PBE on a Nexus 5 running Lollipop. I have seen that there is an issue with the decryption of the data encrypted with a key generated using the PKCS#12 derivation method.
PBKDF2 on the other hand worked fine.
In Detail :
I have tested the same on a variety of devices - running Android 4.0, 4.2.2, 4.3, 4.4.2, 5.0.1, 5.0.2.
I have seen that PKCS#12 works on 4.2.2. It failed on 4.3, 4.4.2, 5.0.1, 5.0.2.
PBKDF2 works on all devices and versions.
Now, what might be the reason for this ? Does it have something to do with the change in the SecureRandom Implementation from BC to OpenSSL from Android 4.2. But that doesn't fit well since I have seen 4.2.2 working.
@Nikolay : Any clues ?
Last but not the least... Excellent Write up!. Thanks a lot for the effort.
Can I use this method as PIN protection to encrypt a Symetric key previously generated and store on device? Like this:
1-Two users configures a password to encrypt/decrypt data to be transmitted
2-My app ask a pin code to protect app(user have to enter each time)
3-Encrypt initial key (1) with this pincode
4-User enter to app, input the pincode, decrypts the stored ecrypted key (1) and read data
How can I check if the decryption of key(1) was succesful?
I used your code. It works perfectly between devices after post-4.2 Android.
There is one problem.
For code encrypted by post-4.2, pre-4.2 decryption generate error "pad block corrupted".
For code encrypted by pre-4.2, post-4.2 decryption generate error "error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt"
Please help