Updated Epiphany Reference Manual

Forum for anything not suitable for the other forums.

Updated Epiphany Reference Manual

Postby aolofsson » Mon Sep 30, 2013 7:14 am

Hi all,

Thanks to notzed and others asking the tough questions, I have decided to publish the updated Epiphany architecture specifications as-is.
The manual still needs editing, but at all the information/tables should be correct.

http://www.adapteva.com/docs/epiphany_a ... .09.29.pdf

You will notice many new multicore features and registers in the specifications. These features were always in the silicon but we never had the bandwidth to properly validate them post-silicon. Much gratitude goes out to our summer interns from WPI (Xin and Wenlin) who worked non-stop this summer to test some of these crazy features. The examples they put together can be found here:

https://github.com/adapteva/epiphany-examples

Cheers,
Andreas
User avatar
aolofsson
 
Posts: 1005
Joined: Tue Dec 11, 2012 6:59 pm
Location: Lexington, Massachusetts,USA

Re: Updated Epiphany Reference Manual

Postby notzed » Tue Oct 01, 2013 2:29 am

Nice - quite a lot of interesting new stuff. The loop thing we knew about, multicast, sync, wand and other workgroup support, the message-mode dma, and the custom routing and core protection in the IV chip.

I noticed you clarified what '!interrupted' means in idle, thanks. And now the sync interrupt name makes sense too.

Of course some more questions spring immediately to mind after a quick browse (sorry!) and might be worth considering for it's release if not already in the pipeline:

How are sync/wand/mbkpt is propagated, e.g. via write mesh or separate oob signal? Is it only on-chip? Which mesh does multicast use/can it work over external links? Are there alignment restrictions/deadlock caveats with gie+wand+idle as per the gie+idle thread? How is the membership of the group specified for wand? (as in 'When all cores in a group have set their respective wand bits').
notzed
 
Posts: 331
Joined: Mon Dec 17, 2012 12:28 am
Location: Australia

Re: Updated Epiphany Reference Manual

Postby aolofsson » Tue Oct 01, 2013 2:57 am

notzed,

Thanks for reviewing the docs!

-sync/mbkpt/wand is only for on-chip groups, no multichip propagation.(no pins left :( ) The signals are completely independent from the eMesh networks

-the multicast works only for write transactions and has been tested for on-chip write transactions and host-->epiphany. We need to test some multichip boards to guarantee that it will work for multi-chip.

-the wand ISR can be tricky and I suspect we may have to look at this more closely.

-group edges are specified in the MESHCONFIG register (bits 12-15)

Andreas
User avatar
aolofsson
 
Posts: 1005
Joined: Tue Dec 11, 2012 6:59 pm
Location: Lexington, Massachusetts,USA

Re: Updated Epiphany Reference Manual

Postby notzed » Tue Oct 01, 2013 5:36 am

Andreas,

Thanks for the details - looks very interesting. I couldn't see how the host cpu could write to a multicast address and configure the transaction for multicast but knowing it's possible is good enough at this point. One last detail, is the sync/etc signal propagated by hops or more direct (i.e. lower latency?) (looks like it could be?).

Now I think i need to spend time playing with it before asking much more.

Cheers,
!Z
notzed
 
Posts: 331
Joined: Mon Dec 17, 2012 12:28 am
Location: Australia

Re: Updated Epiphany Reference Manual

Postby timpart » Tue Oct 01, 2013 6:57 am

In the multicast mode, does each core configured to receive the multicast have exactly one write made to it? Not sure whether you have done this, but makes life simpler. I'm guessing a group has to be rectangular to simplify implementation?

If a DMA is given a multicast address as its destination, would a message interrupt at the end go to all the receiving cores?

Thanks,

Tim

P.S. do you want typos reported here or in the feedback thread?
timpart
 
Posts: 302
Joined: Mon Dec 17, 2012 3:25 am
Location: UK

Re: Updated Epiphany Reference Manual

Postby hewsmike » Tue Oct 01, 2013 7:15 am

I'd read it as : one write to each core that has a matching multicast value, per multicast write from source, by whatever is the write quantum - byte, half-word etc. Such multicasts would 'radiate' to be tested ( for entry to a node ) over the entire chip eventually. When you want to go back to normally targeted writes emitted from a given core then flick multicast off.

Cheers, Mike.
hewsmike
 
Posts: 85
Joined: Mon Dec 17, 2012 3:20 am

Re: Updated Epiphany Reference Manual

Postby timpart » Tue Oct 01, 2013 11:31 am

Mike I agree it should reach all cores set to receive. I'm just wondering if each core only gets the message once. For example if a core is North East of the starting point there are two ways to reach it: North then East or East then North. How do its neigbours behave and does it get the message twice?

Double messaging can be avoided with a appropriate routing protocol. E.g. only send North if message came in from the South and vice versa; always send East and West providing not going back the way you came in. I'm wondering if Adapteva have actually done something like this or not. I'm hoping they might have to avoid flooding the mesh with messages.

Tim
timpart
 
Posts: 302
Joined: Mon Dec 17, 2012 3:25 am
Location: UK

Re: Updated Epiphany Reference Manual

Postby aolofsson » Tue Oct 01, 2013 5:42 pm

Notzed:
To send a multicast/broadcast from the host, you would write to a memory mapped register in the Epiphany FPGA logic (documented in the SDK?) that would force the control mode field sent on the transaction over the eLink to be sent as a multicast transaction.
The sync/wand/bkpt signals get propagated through registers so there is latency. We needed registers to keep the timing synchronous (and fast) across the chip. Think of these signals as two propagating "waves": One in the northwest direction and one in the southeast direction.

Tim, Mike:
The intent was to design the mesh network so that only one transaction gets to each node, hopefully we got it right.
Multicast "listening" at the receiver is always on. The indication of a multicast incoming transaction is indicated by the 4 bit "ctrlmode" field that is sent along with every transaction on the mesh. We use that ctrlmode field to indicate things like messaging as well.
Multicast not available for 64 bit transfers (we need those bits for the source address)
Setting the multicast mode with DMA and messaging won't work.(will add explicit restriction in spec)

Errata:
Currently multicast only works when the transmitter is on row "0". :(

ps. Please don't report typos, there will be too many..will fix it and release new version

Thanks,
Andreas
User avatar
aolofsson
 
Posts: 1005
Joined: Tue Dec 11, 2012 6:59 pm
Location: Lexington, Massachusetts,USA

Re: Updated Epiphany Reference Manual

Postby timpart » Tue Oct 01, 2013 9:59 pm

Thanks for the answers Andreas. I'm guessing multicast is incompatible with TESTSET as well. Can't see how that could possibly work.

Regards,

Tim
timpart
 
Posts: 302
Joined: Mon Dec 17, 2012 3:25 am
Location: UK

Re: Updated Epiphany Reference Manual

Postby timpart » Fri Oct 04, 2013 10:48 am

On page 143 in the description of FSTATUS it says bits 0 to 2 of the STATUS register are read only. This disagrees with page 154 and the hardware barrier code example which also think bit 3 is read only and needs to be changed via FSTATUS.

Tim
timpart
 
Posts: 302
Joined: Mon Dec 17, 2012 3:25 am
Location: UK

Next

Return to General Discussion

Who is online

Users browsing this forum: No registered users and 11 guests

cron