Pcie card with 64 core chips

[marek]

Hi Andreas and co,

Have you thought of making a pcie card with eg 4x4 64 core chips? Something similar to knights corner. Or populate the entire board with 64 core chips of course display port interface is a must!

[sebraa]

Why would you need a Display Port on a PCIe card?

[piotr5]

hi, I have nothing to do with adapteva, but since andreas skipped this posting:

slowly I begin to think a pcie card with many 16-core chips would be better, for the increased bandwidth. of course it'd have to be pcie5.0 with its 15G/s, that's about 16*15 chips and 16 fpga chips with 2G/s each sending data over a specialized pcie-chip which tones down the 30G/s to 15G/s -- provided someone else develops the many-to-single pcie5 protocol chip...

the reason is I've read of , and it seems reducing size of chip-components isn't all that great an idea, especially if it means reducing clock-frequency too for the case of damaged paths on the chip because with reduced wire-size also heat-specs get worse. keep in mind, 16x15 chips will consume between 256W and 500W with over 4k cores, better spread that heat over a large area than introducing whining fans...

[sebraa]

I don't think that 128 Epiphany chips would fit on a PCIe card.
Oh, and while your solution might be able to maximize bandwidth, you would not be able to either program or pay for it.

[piotr5]

not sure how one can program bandwidth. you mean fpga-programming?

as I said earlier on this topic, the actual problem isn't one of fitting 256chips on a pci-card, 8x16 cm formfactor would theoretically allow for cramming 128 of 1cm² chips per side, preferably seperated by some material which doesn't conduct much heat. the real nightmare is with all the wires on the board and the em-chaos they'd cause at 500w (and the signals they might capture when turned off).

and of course it wouldn't be much problem to cool such a >2x128cm² thing either, unlike the 28nm technology of 64-core which would consume only ~64cm² instead (although at slightly less power).

as for costs you are right. a lot of fpga would be needed to handle the 2x16G/s over the thousand wires. also please note that I wrote "pcie5", pcie4 is currently out and not yet well-implemented, so I do expect it'd take time to construct such a beast.

and finally as the article I linked to says: it simply makes no sense to even own a 256-core even less 4k-core beast because no programming-techniques are known to make use of them. that's why we're here, find a way to utilize the 16 cores we have and prove wrong those nay-sayers who based on programs from past claim that beyond 4-8 cores you wont get any efficient speedup on a regular basis. I'm all for maths about inter-core bandwidth, but real programming is always a matter of innovation and not just in extreme situations...

[aolofsson]

@piotr

Appreciate your enthusiasm on this forum, but please refrain from talking about things you clearly know nothing about. It's not helpful.

[piotr5]

sorry, didn't notice my reply was feeding the trolls. (by feeding trolls I mean igniting controversy which wouldn't be necessary since I know very well that what I say is not applicable with current methods of technology.) guess I should have said up front: electronics is something I clearly know nothing about. my argumentation was purely from a point of view based on naked numbers, without taking context into account.

but to undo my reply: epiphany is no competition to knights-corner. from the point of view of programming, epiphany's on-chip memory is limited and external memory is expensive in bandwidth especially for 64-core. programming knight's corner is probably much easier. also complexity of instruction-set differs, as epiphany is optimized for lower power-consumption while intel had ease of use in mind. programming without a cache is difficult even if there is software-cache mechanisms. so in terms of what I do know about there does not seem to be any usecase where epiphany pcie card would actually beat intel. I suspect that also research-/production-costs couldn't be lower than the costs intel has for its own products, with their long experience on the market. you'd need to add other innovations into the final product, innovations intel didn't think of yet. and even then it is questionable if a market for this kind of products will actually exist in future as, again from point of view of programming, the focus is more on cell-phones and remotely running programs on hardware which likely has no pcie slot at all. if you're rich maybe you could try it, circumstances can change and maybe innovation will come to you. but for adapteva the risk of waiting for miracles to happen is too high. they already have a sufficiently successful product...

[sebraa]

[marek]

Too bad the way this discussion turned out to be but my question was more about whether it's possible or feasible to make such a board which could be used both as a gpu replacement (hence the DisplayPort) and for general purpose computation
Sure the memory constraints can be limitating but good programmers have always found ways to go around such obstacles - not to say really good programmers always view it as a challenge. And careful hardware design could minimize these shortcomings.

[marek]

I forgot to add these chips are going to be relevant for the next couple of years without moving a finger.
No need to improve or redesign, what epiphany needs is a good product or two.
One would be the parallella board - single chip mutilple boards
Second would be the pcie card - multiple chips single board
Please if you choose to engineer the pcie card don't forget about the graphics