Peer-to-peer distributed Matlab computing – update

2 Replies

After discussing in detail with colleagues at the Donders and at the FIL, I have implemented the peer-to-peer distributed computing toolbox for MATLAB. Most of the desired functionality is now in place, and it seems to work robustly and efficiently.

The peer toolbox allows you to do something like this in MATLAB

a = randn(400,400);
tic; cellfun('pinv', {a, a, a, a, a}, 'UniformOutput', false); toc
tic; peercellfun('pinv', {a, a, a, a, a}, 'UniformOutput', false); toc

Continue reading →

Peer-to-peer distributed Matlab computing

Leave a reply

In a recent meeting with the SPM developers, we discussed parallel computing using the Matlab distributed computing toolbox, Star-P, Sun Grid Engine, and other batch systems that can be linked to Matlab. These are all limited in their usefulness for the typical neuroimaging research setting in that they are based on a centralized job distribution system. That may work fine on a large cluster with a centralized configuration and system administration, but even then the usefullness is limited because all input and output data (which are typically large) have to be send over the network twice: first to the job manager, then to the compute node (and vice versa for the results).

To resolve some of these problems, I came up with the idea of peer-to-peer distributed computing in Matlab. The full description can be found on http://fieldtrip.fcdonders.nl/development/peer

Automatically compile a missing mex file on the fly

Leave a reply

I am maintaining a few open source Matlab toolboxes. Although most functions are plain Matlab, some functions are implemented as mex file. The mex files have to be compiled on each platform, i.e. 32 and 64 Linux, 32 and 64 bit Windows, ppc and intel Mac OS X, etc. Since I don’t have access to all those platforms, I have devised a trick to automatically compile the mex files on the end-user’s platform.

Let’s say that you have a function mymexfile, then the file mymexfile.c would contain the c-code implementation of the Matlab mex file. You should save the following code as mymexfile.m. If the compiled mex file (e.g. mymexfile.mexw32) does not exist upon the first call to the function, the m-file will be executed. The m-file will automatically compile the c-file and will subsequently call the compiled mex file. On all subsequent calls, the compiled mex file will be executed directly.

function [varargout] = autocompile(varargin)

% AUTOCOMPILE compile the missing mex file on the fly

% remember the original working directory
pwdir = pwd;

% determine the name and full path of this function
funname = mfilename('fullpath');
mexsrc = [funname '.c'];
[mexdir, mexname] = fileparts(funname);

try
% try to compile the mex file on the fly
warning('trying to compile MEX file from %s', mexsrc);
cd(mexdir);
mex(mexsrc);
cd(pwdir);
success = true;

catch
% compilation failed
disp(lasterr);
error('could not locate MEX file for %s', mexname);
cd(pwdir);
success = false;
end

if success
% execute the mex file that was juist created
funname = mfilename;
funhandle = str2func(funname);
[varargout{1:nargout}] = funhandle(varargin{:});
end