I've taken the patch you sent here:

  http://article.gmane.org/gmane.comp.cryptography.viff.patches/54

and split it for you into some smaller pieces as suggested in my
review. They are now pushed as revisions ed3302fd93f1, a8206d017f43
and d5741e841ccc.

At the Amsterdam meeting Benny Pinkas mentioned that a student of his
has been thinking of using the Google Protocol Buffers format too.

That should probably not concern us for our internal communication,
but if we have to exchange data directly with FairPlay it might come
in handy to use the same format.

Quoting Martin Geisler <tracker@viff.dk>:

>
> Martin Geisler <mg@daimi.au.dk> added the comment:
>
> Patch here:
>
>
> http://lists.viff.dk/pipermail/viff-patches-viff.dk/2008-October/000053.html
>
> Comments here:
>
>
> http://lists.viff.dk/pipermail/viff-patches-viff.dk/2008-October/000054.html
>
> As I understood it on Mikkel, the performance is very close to what we
> are used to, some tests were a tiny bit better, some a tiny bit worse.

Even in extreme tests, the differences are quite small. The new version IS
marginally slower (perhaps if it had also been written in C, it could be as
fast), but then again using the other version in the active case would be
cheating.

Patch here:

 
http://lists.viff.dk/pipermail/viff-patches-viff.dk/2008-October/000053.html

Comments here:

 
http://lists.viff.dk/pipermail/viff-patches-viff.dk/2008-October/000054.html

As I understood it on Mikkel, the performance is very close to what we
are used to, some tests were a tiny bit better, some a tiny bit worse.

That sounds very good!

It will be interesting to see the benchmark results for this change.

I have now replaced the marshal module with a more or less hand-coded
version that is not able to do scary things if we get malformed messages.

It also makes what I think is a necessary change to the send/receive
semantics. The receive and send code are now supposed to send data in
the form of a string. That makes sense in my mind at least; the send
and receive code plays with a chunk of data with a program counter
attached and that's it. The rest is up to the sender or receiver - for
example, sendShare now makes a string out of its share and calls sendData.

Before releasing a patch, I want to make sure it does not ruin the
efficiency completely. I have more than a few ways to vary the code,
and the goal is to select the most efficient method.

We might want to look into the newly announced Protocol Buffers format
created by Google:

  http://code.google.com/apis/protocolbuffers/

It can be read and written from Java, C++, and Python. It claims to be
faster and simpler than, say, XML, but that doesn't say much :-)

Added nosy list from Issue 36 and marked this as "urgent".

Issue 36 confirms that the data generated a player running on one
platform cannot be read by a player running on another platform!

Another reason why we definitely want to switch to something else is
that the marshal module doesn't guarantee that the data can be exchanged
between different Python versions or platforms!

The marshal module is actually meant for use by the Python VM when it
generates the .pyc files. And these can always be regenerated if they
are found to be out of sync with the VM running.

Just a note: I proposed the struct module instead of the more standard
(and more powerful) pickle module since the pickle module carries the
same warning as the marshal module.

The marshal documentation can be found here:

  http://docs.python.org/lib/module-marshal.html

We might want to use the struct module for fast reading and writing of
binary data:

  http://docs.python.org/lib/module-struct.html

The documentation for the marshal module says that:

  Warning: The marshal module is not intended to be secure against
  erroneous or maliciously constructed data. Never unmarshal data
  received from an untrusted or unauthenticated source.

So when we start dealing with active adversaries, we should replace it
with something that is secure, probably something that uses the struct
module. See this discussion for more details:

  http://thread.gmane.org/gmane.comp.cryptography.viff.devel/19