Parallella Community

I promised I would get some data later tonight regarding heat distribution on my cluster setup. Here's a pretty graph.

clustertemps-01.jpg (141.37 KiB) Viewed 28690 times

The graph shows the temperature of each board in the stack, with the position in the legend corresponding to the vertical position in the stack. The variation in temperatures generally accompanies different positions that I tried, with each position getting about 30 seconds to normalize. I'll have a more thorough setup in later days this week.

I'm not an engineer, but it looks like for stacked boards the temperature variation depends pretty heavily on board position in the stack and relative position of the fans. This should be pretty obvious (like, really obvious), but the graph somewhat quantifies this.

I will post my scripts and additional experimental data later when I can approach it more seriously, but I need to go to bed right now.

It would be great if you could share your scripts and I'm just wondering how to best to collect together, and perhaps even package, utilities like this. Rather than adding them to the repository, maybe we should have another called parallella-utils, which contains system admin stuff instead of parallel programming examples. These could then be packaged and made available in the default install and updated via a Parallella APT repo.

Then there are also existing system admin and monitoring tools etc. that we could choose to integrate with.

Thoughts?

Cheers,

Andrew

@optimaler: I just love charts! And it looks like you have all 8 boards linked up now.

The stability of a Parallella seems to depend, in part, on keeping temperatures stable and low. The differences shown here for each board are significant - and a tad high - and it shows that this is an important project. Go for it!!!

You say you tried a few things, such as different standoffs, positions of the fans, etc. Hopefully - after you've had some sleep - you'll post a few more pictures with the results.

Your picture at shows two fans stacked vertically. Lots of air pushed - but inefficently.
Comparing with the embecosm stand/approach {as shown by @9600} at , have you tried a push/pull arrangement of the two fans?

Looks like some DIY ducting/casing would help too. Nothing too ugly, of course

@9600 : separate parallella-utils - gets my vote.

tery

Interesting!

Some people can predict these things (temperate simulation is a normal part of PCB design...).

I have more data. I apologize for the long post. TL;DR - Use an enclosure with push/pull fans, the thermal compound on the heatsinks is not thermal compound; using real thermal compound decreases temperatures by up to 4 deg C.

I built a cardstock case to direct the flow of air more cleanly, shown below. The setup is push/pull; I tried to block as many openings as possible. The end with the pulling fan is detachable so I can access the reset buttons (I am still having grief with eth0 not coming up properly).

enclosurecollage-small.jpg (157.43 KiB) Viewed 28581 times

The graph below shows the temperatures with and without the pull fan attached to the enclosure. The transition occurs around 60 s, leveling off around 120 (as best I can tell). Like in my original post, the vertical board position corresponds to vertical position in the legend.

enclosedairflow.jpg (140.41 KiB) Viewed 28581 times

My observations:
1) The enclosure makes the cooling more consistent, and for the most part drops the temps about 2 deg C.
2) Using push/pull instead of just push with the enclosure gives approximately 2 deg C additional drop.
3) I wondered why board 5 was running hot compared to the other boards. The best hypothesis I can come up with is that the chip runs slightly hot compared to the others; I examined the placement of the heatsinks to see if there was any observable difference.

I randomly (and accidentally) pulled one of the heatsinks off my boards. this seemed weird to me, because I thought the silver stuff was a thermal compound. Ha, no, it's some kind of tin foil.

So I conveniently have some Artic 5 silver compound sitting around, so I decided to see what the temperature difference between the two might be. I scraped off the tin foil, cleaned off the heatsinks with 70% isopropyl, and applied artic silver. I put single board in my enclosure, with and without Arctic 5 applied. Here are the results:

thermalcompound.jpg (59.04 KiB) Viewed 28581 times

As you can see, you can decrease the temperature by about 4 C by using proper thermal compound, versus weird silver foil stuff. I don't necessarily endorse this modification of the setup, but honestly it makes a non-trivial difference. Maybe someone in the Adapteva crew can endorse the use of thermal compounds so everyone can enjoy lower (and safer) temps?

At this point, the only experiment left that I can think of to try is longer standoffs, but I don't have any at the moment. I'm not sure how much of an impact that would make, either. Short of liquid cooling, I think this is probably the best anyone can do (although, someone please prove me wrong).

@FHuettig - thanks, we have no real disagreement at all.
The temps ARE well within the 85 degree spec for the Zynq 7000 series, even with the 'stock' 'tin foil' attached heatsinks.
I suppose it's because I have a legacy mentality from being a computer hobbyist for decades. Heatsinks the size of grapefruits were once (are still?) the norm for an overclocked PC.
Still, something tells me that keeping a stable, low/lowish, temperature for any electronics system increases its life and uptime. Every little helps.
And it's a bit ironic that we're talking about a heat issue on an ARM-based chip, when the Parallella is a platform to demonstrate the low power/low heat Epiphany.

@optimaler - good stuff.
Card Stock sure beats 3D Printing when it comes to meeting deadlines!! Hope those are anti-static socks.

Larger standoffs will increase the overall height and you'll probably have to go back to stacked fans - or say, clusters of four per 'box' - or, say, different ducting between the boards. {....which can easilly get out of hand.}

... water cooling? Mmmmmm!

No surprise that properly applied Arctic silver is more efficient than the adhesive thermal tape, but thanks for the example and data. I'd planned on doing much the same.
The issue is how to safely and securely fix the heatsink when there are no holes/clips on the board. Card Stock won't help here, I'm afraid.
From the above, the thermal tape approach is deemed 'good enough' to stay within the spec for the Zynq - so I guess we're in the post sales, modding, hobbyist area as far as Adapteva is concerned.

I also wonder why there is variation between the boards which doesn't seem to tally with board position.
Is it because of differences in the 'tin foil' attachment (my guess), or because the Zynq's production output varies this much (I doubt/hope not)?

Note: My boards will be based on the smaller 7010 array, and I suspect they would have a marginally lower operating temp. Also, like here, I won't be using the HDMI circuitry which would have pushed the temps up higher still.

tery

@gretery, @optimaler: Thanks for the great feedback. The Kickstarter boards are being shipped with the zynq 7020 engineering silicon (a long story...) which has an errata item that makes the Zynq standby (leakage) current higher than it should be (and somewhat variable between chips). We only have partial tracking of this data, but estimate is a delta of 0.1A for the zynq. Of course, leakage current is highly dependant on temperature, so the hotter it gets, the more it leaks..

Andreas