JTAG Programmer only accepts FPGA configuration files in the binary bitstream format (.bit). It does not allow configuration using the ASCII raw bits (.rbt) format.
Express mode bitstreams cannot be used to configure devices via boundary-scan.
Make certain that the boundary-scan (BSCAN) symbol has been included in your design. If it has not then the bitstream will also not be usable for boundary-scan based configuration. Standard examples for instantiating the BSCAN symbol in FPGAs are included in Appendix F.
Keep your device bitstream files separate for each device in the boundary-scan chain. JTAG Programmer requires you to assign a single bit file to each device. It cannot manipulate composite bit files.