To summarize: the salt should always be zero-padded to eight bytes.
This summary is true for user-defined salts created using the 's' switch, because user-defined text cannot create salt strings with embedded 0x00 bytes followed by other not-0x00 bytes. To include the randomly created salts, more accurate would be that the salt should always use the entire 8 byte salt at bytes 9-16 inside the mime string - which the 's' switch matching a salt-parameter shorter than length 8 are being padded to 8 with 0x00's, and salt/iv parameters longer than 8 bytes are being silently chopped to 8.
That said, I am guessing this will break backwards compatibility. But if I understand you correctly, this will make it compatible with the standard implementation.
True.
There are 3 incompatibilities introduced in the recent posts of this thread. For anyone encountering incompatibilities, these workarounds should solve most cases:
1. For syntax which does not use the 'l' switch for a literal key, the input to the hashing algorithm is no longer chopping the key parameter at 56 UTF-8 encoded bytes. This alias should allow most older long text passwords to be used after these fixes.
alias chop_key_to_56 {
noop $regsubex(foo,$1,,,&maroon.tmp)
returnex $bvar(&maroon.tmp,1-56).text
}
$decode(mime_string,mc,$chop_key_to_56(key_parameter) )
$decode(mime_string,mr,$chop_key_to_56(key_parameter) )
$decode(mime_string,mi,$chop_key_to_56(key_parameter),user_iv )
The exception would be any keys where any codepoint 256+ is partly within/beyond the hash algorithm's former chop limit.
//echo -a $chop_key_to_56($str(a,55) $+ $chr(233) )
In this case, 1 of the 2 encoding bytes was used and the other was ignored. Since this used an invalid UTF8 string as the key, there's no workaround other than allowing key/salt/iv to be hex/binvar.
2. The fix where codepoints 256+ in a salt/iv parameter were always used as the '?' character is easy to fix. If this were a user-defined salt created with the 's' switch, the '?' is already in the mimestring's header, so the file can be decrypted as normal without the 's' switch. Same applies to user-defined IV placed into the mime's header using 'mcir' or 'mcirl'. When using 'i' without using 'r', the IV is not placed into the header, so would need to be decoded by substituting the '?' in place of the codepoint 256+'s in the 'i' switch's IV parameter.
3. When the Salt was zero-truncated shorter than 8 bytes before being used, the mime string can be modified to move bytes from the key parameter into the salt, which should allow the new $decode behavior to arrive at the same salted-key+IV used to encrypt the file. There's 2 exceptions which would require hex keys to fix:
a. Depending on where the 0x00 first appears in the salt string, up to 8 bytes might need to be moved from the key parameter to the salt. If the key was not long enough, then the following alias won't be able to cannibalize the key parameter to create the 8 byte salt that's required. My example in post 265381 used the 3 byte string 'key' as the key parameter when creating these example strings, so the only mime strings which could be fixed for the new decoding behavior would be those where the 0x00 was not in the first 8 bytes of the salt.
b. The key parameter is UTF-8 encoded, but the salt is not, so - depending on the location of codepoints 256+ near the end of the key parameter - the key cannibalizing might not be able to create a valid 8-byte salt and a valid UTF-8 text string at the same time.
This alias should fix mime strings encountering issue#3 except for the 2 exceptions noted above:
alias upgrade_salt {
echo -a syntax: //noop $ $+ upgrade_salt(old_mime_string,key_parameter)
var %pattern /^[0-9a-zA-Z/+]+={0,2}/g
if (($len($1) !isnum 32-) || (!$regex($1,%pattern))) { echo -a invalid mime string $1 | return }
bset -t &maroon.tmp 1 $1 | noop $decode(&maroon.tmp,bm)
echo -a $bvar2hex(&maroon.tmp,1-16) $bvar(&maroon.tmp,1-16).text
if ($bvar(&maroon.tmp,1-8).text !=== Salted__) { echo -a this was not created with a random/user salt | return }
if (!$istok($bvar(&maroon.tmp,9-16),0,32)) { echo -a salt does not contant 0x00 and doesn't need to be fixed | return }
var -p %key $2- , %i 9 , %salt_used , %keylen $len(%key)
while (%i isnum 9-16) {
if ($bvar(&maroon.tmp,%i) > 0) var -s %salt_used %salt_used $v1 | else var %i 16 | inc %i
}
while ($numtok(%salt_used,32) < 8) {
var -s %byte $asc($mid(%key,%keylen,1))
noop $regsubex(foo,$mid(%key,%keylen,1),,,&maroon.moved.char)
dec -s %keylen | var -s %salt_used $bvar(&maroon.moved.char,1-) %salt_used
if ($numtok(%salt_used,32) > 8) { echo -a unable to fix due to UTF8 char %a in key | halt }
if (($numtok(%salt_used,32) < 8) && (%keylen == 0)) { echo -a unable to fix due to too-short key | halt }
}
bset &maroon.tmp 9 %salt_used | noop $encode(&maroon.tmp,bm)
noop $regsubex(foo,$left(%key,%keylen),,,&maroon.tmp.key)
echo -a try to decode with: //echo -a $ $+ decode( $bvar(&maroon.tmp,1-).text ,mc, $left(%key,%keylen) )
echo -a if contents of mime are binary you may need to load this mime string into a binvar and decode that
echo -a characters in the key: $bvar(&maroon.tmp.key,1-)
}
If the shortened key contains a trailing space character, you might need to use $+ $chr(32) to re-create the key string shown in the last line of the alias's display.
In the earlier post's example where "//echo -a $encode(message,mcs,key1234,5678)" was creating a key using only the 4 byte salt '5678' instead of also including the 4 0x00's in the mime header, the 7.55-and-earlier output is:
U2FsdGVkX181Njc4AAAAAOZf+ZmPFy3l
To repair this so the new $decode behavior can decrypt it, the mime must be altered to move 4 bytes from the key into the salt:
//noop $upgrade_salt(U2FsdGVkX181Njc4AAAAAOZf+ZmPFy3l,key1234)
The alias modifies the salt string inside the mime and recommends trying to decode with a key shortened from 'key1234' to 'key' :
//echo -a $decode( U2FsdGVkX18xMjM0NTY3OOZf+ZmPFy3l ,mc, key )
Which should work after the fix.
