mIRC Home    About    Download    Register    News    Help

Print Thread
Joined: Jan 2004
Posts: 2,127
maroon Offline OP
Hoopy frood
OP Offline
Hoopy frood
Joined: Jan 2004
Posts: 2,127
Base32 is consistent in how it decodes text with/without padding, but base64/mime using the 'm' switch adds extra 0 bits when non-padding characters aren't padded to be within a group of 4 encoding chars.

Also, when there's no padding and length is any of 9 6 5 3 2 1, it decodes to a length-zero value if using a &binvar, but not when using text.

Code:
//var %text $str(/,9)  | bset -t &v 1 %text | noop $decode(&v,bm) | var %text $decode(%text,m) | echo -a length $bvar(&v,0) $bvar(&v,1-).text vs $len(%text) %text
//var %text $str(/,13) | bset -t &v 1 %text | noop $decode(&v,bm) | var %text $decode(%text,m) | echo -a length $bvar(&v,0) $bvar(&v,1-).text vs $len(%text) %text


Quote:

returns:
length 0 vs 7 ÿÿÿÿÿÿü
length 12 ÿÿÿÿÿÿÿÿÿü vs 10 ÿÿÿÿÿÿÿÿÿü


Base32 decodes either all 8 bits of a character, or none of them, depending on whether the encoding string contains the encoding of all 8 bits of the output. But when there's no padding, Base64 adds either 18 12 or 6 0-bits to complete the group of 24 bits. As a binvar, this can increase the length by 1-3 bytes, but as text you only see the bytes which don't decode as 0x00.

/// is the base64 encoding of 6x3=18 1-bits. If the input string is text and the '=' padding character is added to complete the group of 4, $decode decodes ///= into only the 2 complete 8-bit characters. But removing the = causes it to decode as if the remaining bits of the group of 24 bits were encoded as zeroes, causing the returned string to show binary 11000000, or $chr(192). But if the input string is a binary variable instead of text, since this is one of the 6 listed lengths above, the string becomes $null.

The alias below demonstrates this. The lines in green show where the binvar lines are the same with/without padding. The lines are green for all lengths of base32, but base64 strings are green only when the number of non-padding characters isn't a multiple of 4.

/decode_test m
vs
/decode_test a

If base64 is changed to conform with base32, the length-1 handling of binvar's is actually correct, and $decode(/,m) shouldn't be returning an 8-bit decoding of a 6-bit encoding.

//echo -a $hotp(a $+ $str($chr(32),31),123)
... returns error "* Invalid parameters: $hotp" because as described at
https://forums.mirc.com/ubbthreads.php/topics/263454/Re:_$hotp()_base-32_decoding#Post263454
$hotp identifies this as if a base32 string because it checks that the length is a multiple of 8 before stripping the spaces, and since 1 character base32 strings return $null, it generates an error.

base32 is fine: /decode_test a
base64 has issues: /decode_test m

Code:
decode_test {
  var %i 16 , %switch m , %padlen 4 , %char / , %ratio 3/4 | if ($1 == a) var %switch a , %padlen 8 , %char 7 , %ratio 5/8
  if ($2 != $null) var %char $left($2,1)
  while (%i) {
    var %pad $calc( (800 - %i) % %padlen) , %text1 $str(%char,%i) , %text2 %text1 $+ $str(=,%pad)
    bset -tc &v1 1 %text1 | var %v1 $bvar(&v1,0)
    bset -tc &v2 1 %text2 | var %v2 $bvar(&v2,0) , %color 3
    noop $decode(&v1,b $+ %switch) $decode(&v2,b $+ %switch)
    if ($bvar(&v1,1-) != $bvar(&v2,1-)) var %color 4
    echo %color -a %i padding no. bin $bvar(&v1,1-) actual $bvar(&v1,0) calc $int($calc(%i * %ratio)) $chr(22) text %text1 - > $decode(%text1,%switch)
    echo %color -a %i padding yes bin $bvar(&v2,1-) $chr(22) text %text2 -> $decode(%text2,%switch)
    dec %i
  }
}



Edit: Conclusion.

I'm assuming Base64's behavior should change to be the way Base32 handles input. It shouldn't matter whether there are padding characters at the end, and decoded data should only contain bits which are present in the encoded data.

If backwards compatibility somehow prevents that, I would assume, at the very least, Base64 should not handle binvar input different than text string input, and Base64 binvar input lengths of 9 6 5 3 2 should not return $null.

Last edited by maroon; 23/09/18 09:22 PM.
Joined: Dec 2002
Posts: 5,411
Hoopy frood
Offline
Hoopy frood
Joined: Dec 2002
Posts: 5,411
Thanks for the test scripts.

The base64 routines have been rewritten/tweaked/changed several times since the feature was added to accomodate user requests for changes in behaviour. Ironically, v6.35 passes your /decode_test. However, that implementation can no longer be used.

As with most technical features, I rarely implement algorithms like base64 from scratch because it is always better to use an implementation that is established, tested, conforms to RFCs, and so on.

I tried to find another implementation that passes your /decode_test and tested about twenty different c/c++ implementations from a variety of sources - they all failed. Also, most online decoders failed the test, apart from https://www.base64decode.org (although if you enable live mode, that fails as well). Even Microsoft's built-in API CryptBinaryToString() failed the test.

I eventually found one implementation that passes /decode_test. I will use that in the next beta. However, it is not clear what new side-effects this algorithm will have, so it will need testing, and we may have to revert to the current implementation if it does not work out.


Link Copied to Clipboard