I have tested this myself, and worked here. Anyway, I have re-tested it now, with the exact folders you used, and here's the full code:
Code:
alias mdx_fullpath { return $+(",C:\Program Files\MDXStudio\MDX.DLL,") }
alias mdx { dll $mdx_fullpath $1- }
alias mdxinit {  
  dll $mdx_fullpath SetMircVersion $version
  dll $mdx_fullpath MarkDialog $dname
}
dialog anim1 { 
  size -1 -1 313 203 
  title "Dialog" 
  button "positioner",1001,0 0 0 0 
  button "OK",1,3 5 60 30, ok 
  button "",2,4 34 300 160 
  text "A bunch of text",3,63 6 173 17
}
on *:dialog:anim1:init:0:{  
  mdxinit 
  mdx SetControlMDX $dname 2 animation > C:\Program Files\MDXStudio\ctl_gen.mdx 
  did -a anim1 2 open $scriptdirtest.avi 
  did -a anim1 2 play
}


This assumes, of course, that you have a file called 'test.avi' in your script folder.

For this test, I used filecopy.avi, which comes with the MDX pack.

Did you try working with this file? maybe your test.avi is an encoded file, which can't be played using MDX driver?