A: HPVM implementations run on PC's and one of two types of networking -- Myrinet and Winsock 2 (sockets). The latter interface can be used atop a variety of hardware. However, best performance will be delivered on the Myrinet.

Q: Is the source code for HPVM (or subsystems) available?

A: FM runs on PC's and either Myrinet or a sockets interface. The latter interface can operate over any type of hardware.

A: FM 2.1 supports multiple processes per machine and sharing of network interfaces.

A: The `bwx' program measures one-way bandwidth, and the `latx' program measures round-trip latency. Run the `---s' version on the sender and the `---r' version on the receiver. The arguments are the node number of the other node (as calculated from the network configuration file, the message size in bytes, and the number of times to repeat the experiment.

A: No (although such was the case for previous versions of FM). In FM 2.0, programs can communicate as long as they agree on the indexes into the handler tables.

No, sorry.

A: No. There are some performance considerations, though, due to internals of the FM implementation. In most cases, doubleword-aligned buffers will deliver better performance than word-aligned buffers, which will deliver better performance than halfword- or byte-aligned buffers.

A: One call will succeed, and the rest will return NULL.

A: One call will extract data (if available), and the rest will return as if there were no data to extract.

A: No. FM_send_piece() will automatically call FM_extract() if it's unable to send. This breaks the deadlock.

A: Your handlers are improperly defined. Remember, streamify is not only picky about how handlers are coded, but it's also too stupid to output error messages if it can't completely parse your program. Instead, given unacceptable input, it produces incorrect output. (Think of streamify as a glorified sed script.) Scrutinize your code, and ensure you're following the requirements layed out in section Handlers.

A: Make sure you're linking with `libFM.a' or `libFM32.a' (as appropriate), `libLanaiDevice.a', and Myricom's version of `libbfd.a' and `libiberty.a'. We developed FM 2.0 using version 3.0x of Myricom's software. If you're using a significantly older or newer version, it's likely that some symbols we rely on won't exist.

A: This is the most difficult question in the list. Synchronization should take at most a few seconds after all the processes are running. Any of a number of failings could cause FM to hang during synchronization. A checklist of some things to look for would include:

You should be able to answer "Yes" to all of the above questions. If you still have problems, make sure you can run some of Myricom's test programs, to determine whether the fault lies with FM or your hardware and driver configuration.

A: It's possible that the machines didn't really synchronize. Rather, stale data in the network made one machine think it had synchronized when it really hadn't. Sometimes, running lload 0 on all the machines on the network will clean things up. (Don't run lload while an FM program is running, though; that'll usually kill it.) Also, try starting up your FM programs in a different order and see if that causes them to die before synchronizing. If not, check for bugs in your code.

A: It means that FM received a packet from the network that doesn't look like an FM packet. This probably means one of three things occurred. Either:

1. other machines on the same network are running non-FM messaging layers,(11)
2. you have faulty hardware (almost certainly the SRAM on the Myrinet board--Myricom's had trouble in the past getting good SRAM from their supplier), or
3. you discovered a bug in FM. Please report this to us at `fast-messages@red-herring.cs.uiuc.edu'.

A: FM was unable to access the Myrinet device. Make sure the driver is loaded. (In Solaris, /usr/sbin/modinfo | grep -i myri should list a `myri' driver and an `mlanai' driver.)

A: Fortunately, this situation is rare in "real" applications, although it's common in networking benchmarks. The problem occurs only when all of the following conditions are met:

1. The last message sent is fairly large.
2. The receiving process does not FM_receive() the last message (or, at least, the last big chunk of it), but merely ignores it.
3. The receiving process exit()s after "processing" that last message.

What's happening is that, due to the way that message streams are pipelined across the network, the receiver is exiting before the sender has finished sending the last, large message. As a result, the sender eventually fills the network, and then waits for the receiver to drain the network. Because the receiver is no longer running, the sender waits indefinitely. Probably the easiest solution is for the sender to send a zero-byte "cleanup" message at the end of the benchmark, which should set a completion flag in its handler. The receiver shouldn't exit until that message arrives and the completion flag is set.