Announcing PAL: The Parallel Architectures Library

[aolofsson]

Hi everyone,

We have now shipped well over 10,000 Parallella boards across the globe and it has become abundandly clear that the only way we can really move parallel computing forward is by rewriting the standard development stack from scratch!

This need is much broader than the immediate need for the Epiphany to have better software support. Every expert tends to work in his/her comfort zone. Very few people can be effective as true "true full stack" developers, meaning that they can work all the way from algorithms down to assembly and driver programming. (sure there are the few like notzed and patc but they are the exception rather than the rule). Time after time, I have heard...Parallella is a great platform, if only it had a fast XXX... (FFT, BLAS, linpack, ray tracing, openCV, etc, etc).

This is why we started a "from scratch" parallel library effort called PAL in december.

https://github.com/parallella/pal

Please check the README file in the github repo for a complete description and contribution guidelines.

PAL is not designed to be an Adapteva or Epiphany specific project (what the hell do I know about SW ) It's designed to be a parallel computing community effort. With ~10,000 Parallella developers and 200 University collaborators certainly we could make an incredible difference if everyone got involved!

Until now we have kept PAL quiet (but working in the open). It now feels like we have a solid base and we are ready to ask for broader feedback and active contribution. This week we already got number of contributions from Mateusz Kacprzak, which is incredbly exciting since we never reached out directly to Mateusz! If we can get 1% of all of the developers to contribute time, we should be able to bang this out in no time. All it takes is for 100 people to contribute 1 function each!

So for the sake of the Parallella project, PLEASE help. Adapteva will continue to contribute time (and money) to the PAL project as long as we can, but it's only going to be great if we get more people involved. One function per person is all I am asking for...

Sincerely,
Andreas

[piotr5]

so you suggest the program defines a group of programs to be sent to the parallell cores, but what if the programmer wants to define a particular constellation of cores? i.e. there is a graph, each node represents a core, and each edge has a value representing how frequently these cores will have to communicate. this input should then be taken into account when assigning the physical cores to the programs the programmer requested for sending. and this must happen during compilation (if the graph is known then) or during run-time on the host, or at runtime on the parallell programs if they attempt to set up other cores for work!

has this thought been taken into consideration when designing this lib? is it actually possible to run this lib on epiphany which doesn't have the notion of "files"? how do you load a program from shared memory into your neighbour epiphany core with this lib? so far I have not seen any functions dealing with dynamic loading, i.e. parsing the elf-format and adjusting the addresses to the destination-address. are they to be implemented as part of the libc functions?

as for the c++ wrappers, wouldn't it be better to create c-wrappers and program in c++ from the start? would probably take up less space for c++ programs and be easier to port to the other languages. still, in my opinion, it should be programmed in c and the c++ wrappers should be developed in a seperate lib as to extend and improve the original functionality. hell to keep syncronized if the designs diverge -- if you add a test-suite which compares c functions with their c++ implementation. but also a good motivation for the c-functions to stay close with the spirit of object orjented programming. the point of a good c++ implementatiion is so that you can include parts of it in libboost (i.e. parts which already are there but don't support epiphany yet) and thereby reach a bigger community at once. people learned a particular interface, so better offer wrappers which mirror that interface or these people will be reluctant to re-learn the same stuff for your lib anew!

[KOG91]

DataDrake from the KnowmAtRIT fork here.

We are a partnership between Knowm and the Rochester Institute of Technology . Our current focus is on improving the API in order to make the barrier to entry lower for new devs. I have three or so developers (and growing) who will be participating in our fork of PAL.

Looking forward to a bright future with Parallella!

[aolofsson]

DataDrake,
Awesome. Look forward to finding out more!
Lots of people now involved behind the scenes. The sooner you can share some of your thoughts the sooner we can build consensus around the path forward.
Andreas

[mikebell]

I think what you're proposing is basically a library of functions to do operations on arrays of values (for use in programs destined for Epiphany-style cores). Then you have an interface that looks basically the same as the Epiphany SDK to upload programs to cores and schedule them.

That's all well and good, but I'm not sure it massively lowers the barrier to entry for programmers that don't want to worry about scheduling processes across cores.

It would be cool if we could start building up a library and execution environment where the "do this parallel operation on an array" operations were written on the host in something resembling C code (but maybe executed on a VM, as it would have to reorder/parallelize operations to take full advantage of the many available cores). These operations would be seamlessly executed across multiple cores, with sensible scheduling that tries to keep work on the same arrays scheduled on the same core(s).

I haven't done my research, so I don't know if similar efforts exist anywhere. But what I'd like to see is the ability to write code like:

float_vec a, b, c, d, e, f, g, r;
p_abs(b, a); // for (i=0; i<vecsize; ++i) b[i] = fabs(a[i]);
p_add(e, c, d); // for (i=0; i<vecsize; ++i) e[i] = c[i] + d[i];
p_sub(r, f, g); // for (i=0; i<vecsize; ++i) r[i] = f[i] - g[i];
p_madd(r, r, b, e); // for (i=0; i<vecsize; ++i) r[i] = r[i] + b[i] * e[i];

The abs, add and sub operations should automatically happen in parallel across the Epiphany cores, with data movement handled under the covers - I as a programmer wouldn't have to worry about the number of cores available, compiling separate task programs for different architectures, uploading programs to cores, etc.

Then I guess once we had that you could write a higher level language on top - as now these parallel ops are looking a lot like assembler, just where the registers are vectors

Some food for thought I hope!

Mike

[piotr5]

what you suggest sounds very much alike to the Boost.accumulator library. therefore the proper language to do these things is c++. that's why I said we need a separate library for c++, to implement the interface offered by boost and std-lib with the use of epiphany.

that said, about c++ I'm not sure how such a thing could be done: you define a function object, but it physically exists on epiphany while the handle to it is stored on host. that means during compilation a new file needs to be created containing that function and then e-g++ would need to be executed as part of compiling the host program. guess that's beyond the meta-language capabilities of current c++ standard. should we make efforts to change the standard? or has anyone a better idea how to keep the program together in the same file even though it's sent to different cores? thinking of it, maybe just the ability to write a new file from within c++ compilation would be enough, and the build-system (make-files and such) will take over the rest...

[sebraa]

The problem with current Epiphany is not that the calculations are slow, but that the bandwidth to and from the accelerator is too small. Any kind of "please execute ((this single statement)) for each array element in parallel" is not going to solve this problem, unless "((this single statement))" is very involved and takes much longer than up- and downloading the array.

[piotr5]

actually I don't believe communication to epiphany is that slow, although I can't disprove it. but I agree that some form of stacking multiple commands is needed with such an approach. it just doesn't make sense to write p_add twice since then the data would be transferred twice and a lot of time would be wasted. instead you'd need to say "execute p_add twice" and therefore mikkebell suggested some kind of virtual machine language. and actually a virtual machine language is the only way I know, current c++ could reliably use epiphany for optimization, based on a single source-file, with a drop-in replacement for Boost.accumulate interface or its c++ standard lib version.

as for slowness of data-transfer, I guess such a library would then also need to time any data transfer well. if data is sent to a core and also to the core to the right/left, the two transfers will get into eachother's way (although I'm not sure if conversely the second transfer is actually possible at the same time as some other row receives its data). but if some data is shared between them, passing on data from core to core instead of sending twice might be a good idea. this kind of optimizations would then need to be taken into account.

[mikebell]

I guess that's why I'm saying the fine-grained operations would need to be run by a VM that understands where the data is currently local to. So if you did
p_add(a, b, c);
p_add(a, a, d);
The VM would schedule the second add to happen on the same core as the first, and should DMA in the contents of d while b and c were adding.
If the arrays were sufficiently large, the contents and operations could be split across multiple cores.

Maybe this could be a second stage of PAL - first we build up a set of library C operations with low code size that can be formed into Epiphany programs (as Andreas has suggested).

Then when that is available, we could build a VM to run on the host that could intelligently run those building blocks (copying both instructions and data to the cores as appropriate) without the programmer having to explicitly compile individual programs for each set of operations they want to schedule on the cores.

An initial version of this would simply use a .elf program for each operation type, reading arguments from known on core memory locations, and a proof of concept version of the scheduler would copy the arguments in, start the program, wait and then copy the results out. Next we set up indirection for each argument, and add the ability for the scheduler to notice if one of the arguments in the next command was already locally available on chip, and if so use that without performing the copy. From there we can add caching of the code fragments, maybe on the fly compilation of small chains of programs to run on the cores, and the scheduling can become much more complex, reordering operations and intelligently choosing which core(s) to use for which instructions to reduce data movement.

[aolofsson]