parallella-hw fpga project structure - why?
Posted: Sun Sep 21, 2014 5:24 amTL;DR - Why is the repo structured the way it is? Can someone suggest a good workflow actually compatible with it? Now it may be that I am being blind and missing something and really making it hard for myself when really it is not so hard at all - if so, please lead me to the light. If not, I would like to propose a "better way", and see what issues people might see with that approach.
Disclaimer: I am not an FPGA designer, so it is possible these workflows I am rubbishing are industry standard practices - if so, please pardon my ignorance. I'm just saying they don't seem logical to me, planAhead exacerbates the problem, and I want to understand how to work with them properly.
For my own projects, I have restructured things in a way more logical to myself, using mercurial rather than git, which has worked well for me. This was in part because I far prefer mercurial to git - but also I didn't like the way the projects were structured.
So I've been putting off reconciling my workflow with the official repo - because I have something which is nice and works for me. However, now I'm wanting to give stuff back, and moreover I am writing a tutorial(s) to help others getting started - so it is important I make my peace with whatever the recommended flow is so as not to muddy the waters and raise the barrier to entry. Or, since perhaps I can make things easier, I wanted to at least suggest the solution which works for me.
The main problem I have is the way the XPS files (in particular, system.{mhs,xmp}) are located in the edk folder, with an edk folder existing for each project (each having it's own pcores). So in order to copy a project, you need to not only copy the project under fpga/projects (which you can do from planAhead), but also a project edk structure under fpga/edk (which requires a manual process - plus manual hacking of the project xml data to point to the copy). If you copy the project from within planAhead, it will not dereference the fpga/edk, and you will end up cross-contaminating projects (they all use the same XPS files - a recipe for disaster!).
It seems more logical to me that each project has its XPS files in its sources/system folder, which configures the system correctly for the purposes of that project, and determines which pcores are used - the pcores themselves remain in the edk folder, but there is no need for a project substructure to the edk folder - since various projects resuse different combinations of pcores.
Every project I have made thus far, I have needed to alter the mhs file, since I've generated and used a different pcore. And while pcores can be re-used, the combination of them, and how they are configured / mapped is unique to the project. And this is without even reconfiguring EMIO as I will be doing for my robotics projects via the Porcupine.
As I see it, planAhead has two possible behaviours when copying a project (save project as):
Option 1: don't do import all (dereference nothing outside source dir)
Good: doesn't create local copies of global includes in hdl/constraints, and pcores from edk, these remain as link.
Bad: doesn't reference system.mhs/xmp, so issues as described above with cross-contaminating projects, and git pulls start failing because you have edited the Adapteva projects and have local changes - beyond the first project, you'll start breaking past projects by changing a shared copy of the XPS files.
Option 2: import all (dereference everything outside source dir)
Good: no issues with cross contamination
Bad: creates needless copies of everything. This is a nightmare if you like minimal duplication in your source tree - plus it has a very verbose subtree structure for all these things it pulls in which is very ugly. Try navigating this to find the useful files to add to version control. I absolutely loathe this approach.
The way I have restructured my repo allows copying projects using planAhead with option 1 with no issues. The reason this works is the XPS system folder becomes a subfolder of the source dir. So when you copy a project, and it dereferences nothing outside source dir, this is no longer an issue - the XPS files are inside the source folder and get dereferenced.
In practice, (despite having a workflow which works with the UI, I am a commandline hack) I prefer doing an hg copy and fixing up one or two entries in the project file. But I'm happy to test this if you think it is a workflow worth pursuing.
Is there a bigger view here I have missed? What is wrong with my proposed approach?
I can just see my plan to write a tutorial degenerating into a treatise on working with the difficult repo structure. Because I fear the alternative would be negligently ignoring the fact that following the steps will guide people into messing up their repo and having their projects start to break.
Disclaimer: I am not an FPGA designer, so it is possible these workflows I am rubbishing are industry standard practices - if so, please pardon my ignorance. I'm just saying they don't seem logical to me, planAhead exacerbates the problem, and I want to understand how to work with them properly.
For my own projects, I have restructured things in a way more logical to myself, using mercurial rather than git, which has worked well for me. This was in part because I far prefer mercurial to git - but also I didn't like the way the projects were structured.
So I've been putting off reconciling my workflow with the official repo - because I have something which is nice and works for me. However, now I'm wanting to give stuff back, and moreover I am writing a tutorial(s) to help others getting started - so it is important I make my peace with whatever the recommended flow is so as not to muddy the waters and raise the barrier to entry. Or, since perhaps I can make things easier, I wanted to at least suggest the solution which works for me.
The main problem I have is the way the XPS files (in particular, system.{mhs,xmp}) are located in the edk folder, with an edk folder existing for each project (each having it's own pcores). So in order to copy a project, you need to not only copy the project under fpga/projects (which you can do from planAhead), but also a project edk structure under fpga/edk (which requires a manual process - plus manual hacking of the project xml data to point to the copy). If you copy the project from within planAhead, it will not dereference the fpga/edk, and you will end up cross-contaminating projects (they all use the same XPS files - a recipe for disaster!).
It seems more logical to me that each project has its XPS files in its sources/system folder, which configures the system correctly for the purposes of that project, and determines which pcores are used - the pcores themselves remain in the edk folder, but there is no need for a project substructure to the edk folder - since various projects resuse different combinations of pcores.
Every project I have made thus far, I have needed to alter the mhs file, since I've generated and used a different pcore. And while pcores can be re-used, the combination of them, and how they are configured / mapped is unique to the project. And this is without even reconfiguring EMIO as I will be doing for my robotics projects via the Porcupine.
As I see it, planAhead has two possible behaviours when copying a project (save project as):
Option 1: don't do import all (dereference nothing outside source dir)
Good: doesn't create local copies of global includes in hdl/constraints, and pcores from edk, these remain as link.
Bad: doesn't reference system.mhs/xmp, so issues as described above with cross-contaminating projects, and git pulls start failing because you have edited the Adapteva projects and have local changes - beyond the first project, you'll start breaking past projects by changing a shared copy of the XPS files.
Option 2: import all (dereference everything outside source dir)
Good: no issues with cross contamination
Bad: creates needless copies of everything. This is a nightmare if you like minimal duplication in your source tree - plus it has a very verbose subtree structure for all these things it pulls in which is very ugly. Try navigating this to find the useful files to add to version control. I absolutely loathe this approach.
The way I have restructured my repo allows copying projects using planAhead with option 1 with no issues. The reason this works is the XPS system folder becomes a subfolder of the source dir. So when you copy a project, and it dereferences nothing outside source dir, this is no longer an issue - the XPS files are inside the source folder and get dereferenced.
In practice, (despite having a workflow which works with the UI, I am a commandline hack) I prefer doing an hg copy and fixing up one or two entries in the project file. But I'm happy to test this if you think it is a workflow worth pursuing.
Is there a bigger view here I have missed? What is wrong with my proposed approach?
I can just see my plan to write a tutorial degenerating into a treatise on working with the difficult repo structure. Because I fear the alternative would be negligently ignoring the fact that following the steps will guide people into messing up their repo and having their projects start to break.
