Forums Bug Reports $hotp() base-32 decoding

Update: Changed how key parameter is parsed between the 3 rules, and more details.

https://soeithelp.stanford.edu/hc/en-us/...d-or-iPod-Touch

This refers to additional evidence that Google Authenticator format is base32 length 16/26/32 not 16/24/32. It describes the key being 26 characters long. All the examples I've seen describing the GA key being separated by spaces use the example for lengths 16 or 32 where spaces are used to split the key ONLY into groups of 4 digits, but I'm guessing that the format for the rarely used 26 characters of a 128 bit key would do the same thing, except having a final group of the leftover 2 characters. i.e.:



16/26/32 are the lengths of a base32 string that would be encoding a binary key of length 80(half-sha1)/128(md5)/160(full-sha1). The hex lengths 40/64/128 are the lengths for the hash digests of sha1/sha256/sha512. The binary keys encoded using hex should not have 0x00's stripped or the remaining bytes UTF-8 encoded, just as the keys encoded by base32 don't.

Further investigation of $hotp and $totp shows me that it accepts spaces in hex strings as well as base32, as long as the length of the input string including the spaces is one of the lengths 40/64/128. I have yet to find any other program where spaces are included in the length prior to checking whether the string is of the correct length but then thrown away before decoding the key. I still can't find other programs using spaces in any manner other than to chop the 6 output numbers into 2 groups of 3 numbers, or chop a very long input string into groups of 4 base32 characters, where the space is never preced by anything except exactly 4 base32 characters. I can't find examples of spaces used with hex, but if it does exist i can't see it used in any way other than creating the same groupings of 4 digits for ease of data entry with fewer errors.

https://npm.taobao.org/package/authenticator is another url I've found using spaces mixed with base32, and again the spaces are preceded by exactly 4 base32 digits. The webpage uses the example key "acqo ua72 d3yf a4e5 uorx ztkh j2xl 3wiz" which is the base32 encoding of a 160-bit binary key. $hotp and $totp would use this as a literal case-sensitive text key instead of decoding to the 160-bit value, because the string including the 7 spaces has length 32+7=39, and isn't a multiple of 8.

-

I've been able to duplicate how $hotp calculates the output number for all inputs I've tested, except when the key parameter is length 40/64/128 and contains an odd number of hex digits and an odd number of spaces. What is the actual decoded binary key for a string of 39 0's and 1 space?

Because $hmac appends 0x00's to strings shorter than 512 bits, all $hotp keys consisting entirely of 0x00's yield the same output string.



The above key has length 40, so it decodes the key as if it's a hex string of all 0x00's, and returns the same output as 64 0's or 128 0's. If an EVEN number of 0's are changed to spaces while retaining the 40 length, the output remains the same because 20 0x00's and 19-or-fewer 0x00's are both padded to identical HMAC strings. However if an ODD number of 0's are changed to spaces, the output changes to something different, and remains that same different output regardless which is that ODD number of spaces or where those spaces are located within the key string. This tells me that $hotp is not handling the un-paired digit by prepending or appending a '0' to the string, or else 1 space with 39 0's would output the same as 40 0's. The output for 39-zeroes + 1 space is also not the same output from decoding a base-32 string of 39 0's either.

Other than the unknown handling of hex strings containing an odd number of spaces, my prior list of how $hotp and $totp strings are handled changes slightly to:

Rule #1. Current Rule:
a) if key is a string length 40 or 64 or 128
b) and is composed of hex digits and/or spaces
the spaces are removed, and the remaining pairs of hex digits are decoded to byte values 0-255. Value 0 is not added to the decoded key, but otherwise the UTF-8 encoded of each individual value is added to the binary key instead of the actual byte value.
i.e. key matches $regex(key,^[a-f A-F0-9]+$). This means a string of length 40/64/128 consisting entirely of spaces outputs the same value as when those strings consist entirely of 0's: //echo -a $hotp($str(0,40),9) $hotp($str($chr(32),40),9) $hotp($str(0,64),9)
As mentioned above, I'm unable to determine how these strings having an odd number of hex digits are handled.

Fixed Rule:
a) spaces should be removed before checking if string length is 40/64/128.
b) Spaces can only be present if preceded by exactly 4 hex digits.
c) when spaces are removed, string has 40/64/128 case insensitive hex digits.
i.e. key matches $regex(key,^([a-fA-F0-9]{4} )*[a-fA-F0-9]+$) and $istok(40 64 128,$len($remove(key,$chr(32))),32)
Hex strings should be decoded the same way base32 strings are handled, where bytes are not UTF-8 encoded and 0x00's are not stripped. Actual binary key will always be exactly 20 or 32 or 64 bytes in length.



Rule #2. Current Rule:
a) if string doesn't matching #1, but is length 16 or greater that's a multiple of 8. i.e. can be length 40/64/128 if contains at least 1 of the base32 characters not present in the hex alphabet.
b) and is composed of case-insensitive base32 digits and/or spaces
i.e. key matches $regex(key,^[a-z A-Z2-7]+$).
If it matches both requirements, the spaces are stripped and the actual key is the base32 decoding of the remaining string regardless of length.
Rule#2 is not able to handle keys consisting entirely of space characters, even though Rules 1 and 3 can. I'm guessing this is caused by an internal passing of the key to $decode() which does not accept $null strings. $hotp($str($chr(32),N),9) is valid syntax as long as N is not a multiple of 8 greater than 8 other than lengths 40/64/128.
Rule #2 currently has the problem where all text sentences with spaces but having no punctuation, whose length is a multiple of 8, are falsely matched as if they're a case-insensitive base32 string.

Fixed Rule:
a) if string doesn't match rule #1, spaces should be removed before checking if string length is 16/26/32, or optionally also extended to all multiples of 8 above length 8.
b) Spaces can only be present if preceded by exactly 4 case-insensitive base32 digits.
c) when spaces are removed, string is only case insensitive base32 digits.
i.e. key matches $regex(key,^([a-zA-Z2-7]{4} )*[a-zA-Z2-7]+$)
It's debatable whether some false positives should be avoided by requiring the base32 string to be entirely upper or entirely lower case.

Rule #3. Current Rule: Any remaining string is used as the literal case-sensitive key, assumed to already be UTF8 encoded.

Fixed Rule: No change, except many strings change between being handled as hex/base32 encoded vs being handled as literal text.

-

To fix all these HOTP/TOTP problems and preserve backwards compatibility, there would need to be an additional switch to identify the type of key present. There are currently several ways where a key intended to be handled one way is handled another way. The most common ones are:
1. Text passphrase of length divisible treated as if case-insensitive base32 string:

2. Base32 key divided by spaces into groups of 4 characters is treated as if case-sensitive text string because its length isn't a multiple of 8: //echo -a $len(aaaa bbbb cccc dddd)

The additional parameter would default to current behavior, but would allow a switch to force ambiguous keys to be handled correctly.

$hotp(key, count, hash, digits)
$totp(key, time, hash, digits, timestep)
->
$hotp(key, count, hash, digits, type)
$totp(key, time, hash, digits, timestep, type)

type 0 or parameter not present = backwards compatible current behavior

type 1 = key is &binvar

type 3 = Using fixed versions of rules 1/2/3 to determine how to handle the string.

type h = force handling as hex encoded regardless of length. Remove any spaces present, and return error if any non-hex-digits remain or if the number of hex digits isn't an even number greater than zero. Output includes 0x00's and is not translated from binary byte to UTF-8 encoding.

type a = force handling as base32 string regardless of length. Remove any spaces present, and return error if any non-base32-digits or non '=' padding remain.

type m or type u = same as type a, except handling as input to $decode(key,m) or $decode(key,u).

type t = force handling as Rule#3 case-sensitive literal text.

Edit: /help says key is required, but //echo -a $hotp(,9) is the same as $hotp($str(0,40),9)