Using More Memory

Discussion about Parallella (and Epiphany) Software Development

Moderators: amylaar, jeremybennett, simoncook

Using More Memory

Postby eoghanoh » Wed Apr 09, 2014 12:00 pm

Hello,

by default, there is 32MB of shared DRAM available to a program on a Parallella board, even though there's a total of 1GB on the board and there's lots free.

I'd like to use more, e.g. 512MB.

How do I do this?

I presume I need to edit the bsp and / or hdf file or something like that but not really sure where to start. Is there a simple set of instructions anywhere? It seems like something that people will want to do. From everything I've read it seems that Parallella reserves a block of memory when I load the ehal, so I suppose really what I'm trying to do is increase this block of memory so I can access it with e_alloc.

Am I right about this? Can it use more? If it can, a further question is then: why does it ship with only 32MB configured?

Thanks,
Eoghan.
eoghanoh
 
Posts: 23
Joined: Mon Dec 17, 2012 3:22 am

Re: Using More Memory

Postby ysapir » Thu Apr 10, 2014 10:37 pm

There's two parts for this issue. Theoretically, there is no prevention for the Epiphany to access ALL of the DRAM on the board. Technically, this is really a shared, flat memory space. However, due to the nature of Linux, we had to guarantee that no system memory is being overwritten by the Epiphany programs. Thus, we excluded a part of the DRAM from the system's memory map. In effect, whatever the Virtual Memory Manager decides to use, it will not be on that excluded space, and it is safe for the Epiphany to fully use this area.

It was decided quite arbitrarily (well, not really arbitrary - as it was inherited from our previous prototype boards) to allocate 32MB of the DRAM for that purpose.

To change this amount, you first need to change the Linux configuration files (unfortunately, at this time I cannot help with that, but IIRC, it was already discussed in the forums). Then, you need to make the eHAL and eLIB aware of these changes. This is done using the *.hdf configuration file used by e_init().
User avatar
ysapir
 
Posts: 393
Joined: Tue Dec 11, 2012 7:05 pm

Re: Using More Memory

Postby greytery » Sat Apr 12, 2014 5:15 pm

Woo-Hooooo!!!!!

I think this is a very important and exciting topic in its own right.

So, just how fixed is the Parallella memory map?
Is it REALLY possible to transform the E16 from a 16 core * 32KB Duckling into a 16 core * 1MB Swan?

The answer from ysapir above reminds us that the local memory constraint of 32KB per Epiphany core is a Parallella system limit and not an Epiphany limit. The Parallella allocation of DRAM for the Epiphany E16 is a legacy choice - one that was wholly understandable(!) when trying to get the product (and eSDK) off the ground.

The Epiphany datasheet says that the memory architecture allows each core up to 1MB of local memory as a slice of the address space. (Other parts of the address map are reserved so thare may be more unuseable addresses - I'm still working on that part of the manual :? ).
That local memory is comprised of 4 sub-banks which optimally are split as 1 for code, 2 for data and 1 for stack space.

If that 32KB local memory limit can be changed all the way up to 1024KB, then the Parallella becomes a completely different beast:
    Bigger program space per core - code memory bank size up to 256KB.
    Larger Datasets - data memory up to 512KB per core.
    Less pain and grief trying to fit applications on to the limited space.
    It changes Epiphany kernel size limits at a stroke!!
    There's still loads of room for Ubuntu if required.
I can't see anything in the reference manuals that suggests that the efficiency of memory access to a slice of 1MB of local memory per core is any different to the current 32KB slice, so a configuration change to the default local memory size sounds attractive.

Of course, all this cannot be achieved at a stroke - there's a ship-load of configuration changes to be made (and tested).

But WoooHoooo! C'mon folks!!

tery
tery
User avatar
greytery
 
Posts: 205
Joined: Sat Dec 07, 2013 12:19 pm
Location: ^Wycombe, UK

Re: Using More Memory

Postby timpart » Sat Apr 12, 2014 9:28 pm

@greytrey I think you may have misread the postings. The original question was about the 32Mb of the 1 Gb external DRAM that is excluded from Linux general use and available as a shared resource for the ARM and Epiphany cores to use. It is possible to change this allocation so the Epiphany can have more RAM to work with.

However the local RAM on the Epiphany is still fixed at 32Kb per core. The external RAM in the previous paragraph is off chip, and although it transparently forms part of the Epiphany memory map, it is slower to access. (The rest of the 1 Gb is also accessible but since Linux might do anything to it at any time, doing so is dangerous.)

Tim
timpart
 
Posts: 302
Joined: Mon Dec 17, 2012 3:25 am
Location: UK

Re: Using More Memory

Postby notzed » Sat Apr 12, 2014 11:52 pm

timpart wrote: (The rest of the 1 Gb is also accessible but since Linux might do anything to it at any time, doing so is dangerous.)


If you ran a non-paging no-virtual-memory os (uclinux?) then that wouldn't be the case. But really the epiphany or the fpga interface needs a mmu to be useful outside of embedded applications.
notzed
 
Posts: 331
Joined: Mon Dec 17, 2012 12:28 am
Location: Australia

Re: Using More Memory

Postby greytery » Sun Apr 13, 2014 8:52 am

:oops:
@timpart - that's a very polite RTFM. Thankyou.
Got a bit carried away there.
Ho Hum, back to the manual until I get the real thing to play with.
tery
tery
User avatar
greytery
 
Posts: 205
Joined: Sat Dec 07, 2013 12:19 pm
Location: ^Wycombe, UK

Re: Using More Memory

Postby shodruk » Sun Apr 13, 2014 12:42 pm

I think there are some limitations in external memory access.
With the default configuration of Parallella,
the Core-ID of Epiphany is set to (32-35, 8-11),
so the Epiphany can only access its east address (0-63, 12-63).

e.g.
0x00c00000 - 0x03ffffff
0x04c00000 - 0x07ffffff
0x08c00000 - 0x0bffffff
...
0xfcc00000 - 0xffffffff

So we can get up to 52MB contiguous memory space. (with E16 Parallella)

Note: This is only my understanding...
Shodruky
shodruk
 
Posts: 464
Joined: Mon Apr 08, 2013 7:03 pm

Re: Using More Memory

Postby aolofsson » Sun Apr 13, 2014 1:38 pm

Shodruky, That is correct.

Question for all: How big of an issue is it to not support large contiguous areas of physical memory for software developers? Our assumption was that that this could be hidden from most users.

Andreas
User avatar
aolofsson
 
Posts: 1005
Joined: Tue Dec 11, 2012 6:59 pm
Location: Lexington, Massachusetts,USA

Re: Using More Memory

Postby shodruk » Sun Apr 13, 2014 4:31 pm

I think that is not a serious limitation.
It is even possible to implement virtual memory in software.
In any case, with Epiphany, we have to manage data segmented into 8KB or so. ;)
Shodruky
shodruk
 
Posts: 464
Joined: Mon Apr 08, 2013 7:03 pm

Re: Using More Memory

Postby greytery » Mon Apr 14, 2014 11:07 am

{OK, I've calmed down, doubled my medication...}

It's a truism that if an application dataset is larger or differently organised than the limits/constraints of the Parallella architecure then that application will need to work around/within the constraints, such as the 8KB segmentation mentioned.
It's probably a only small subset of applications/developers that really need to access large contiguous datasets in the current Parallella's global memory - so not a big issue.
For those who do need it, perhaps it would be best to just document how to adjust the size and location of the global memory more flexibly/easilly - especially for a E64 build, say, -rather than have it fixed at 32MB. { ... like one of shodruck's Ubunu tips maybe?}

But having access to a large global memory and then actually using it is not straightforward.
The performance hit of accessing the off-chip memory is itself a major constraint - and many developers will have to take this into account even for basic applications.
There's a good analysis/advice by @tnt on the performance impact of reading data/running code from the external global memory on one of the early boards.

http://forums.parallella.org/viewtopic.php?f=23&t=108.

Basically - don't. Unless you must. :(

tery
tery
User avatar
greytery
 
Posts: 205
Joined: Sat Dec 07, 2013 12:19 pm
Location: ^Wycombe, UK

Next

Return to Programming Q & A

Who is online

Users browsing this forum: Majestic-12 [Bot] and 5 guests

cron