Build and Test Automation for non-PDE Eclipse Projects.
At Cape Clear we use PDEBuild, controlled by CruiseControl driven Ant scripts to build our Eclipse features. As Oisín Hurley pointed out recently it has got a steep learning curve but once it is up and running it allows development teams to contribute to features by doing what is natural - using the Eclipse workbench tools to create and modify plugin projects.
Where PDEBuild wins big time is in eliminating redundant build metadata in your build system. The developers workbench is in sync with your build server because they both work off the same Eclipse project metadata. About the only pilot errors that trip up our builds occasionally are
- missing entries in the plugin/bundle build.properties files cause PDEBuild to leave resources out of the packaged jar (or folder). Unfortunately these missing resources don’t always get noticed until runtime. e.g. missing icons showing up as red boxes because the icons didn’t get packaged.
- missing entries in the PDEBuild map file cause PDEBuild to fail to resolve bundle dependencies. If it can’t find the source it can’t build the bundle!
- missing entries in the feature.xml (for a newly contributed plugin/bundle). PDEBuild will only build the plugins listed in the feature!
That’s all well and good for us Eclipse plugin developers. But what about building other types of projects? Headless builds of Java projects (not plugin projects) can be achieved by using the AntRunner application entry point in conjunction with an Ant build script. Unfortunately the Ant script needs to be pre-generated (or hand-written) so one end up back in the situation where the Ant build script can easily get out of sync with the project metadata. For large project sets (>10) this type of duplicate build metadata is difficult and frustrating to maintain.
The issue becomes larger when one considers that many Eclipse based products implement additional natures and builders that they add onto the projects they create. For example, a builder might need to generate some java code or verify spring configuration files). Theses projects need to be built inside Eclipse (since the builders are workspace/workbench dependent) but currently there doesn’t seem to be any facility to do so in a headless manner. I’m thinking along the lines of an application entry point that can perform the following tasks:
- Optionally import a set of projects into the target workspace
- Optionally clean all projects
- Build all projects in the order determined by the declared project dependencies (in the project descriptor), running all builders as it would if the Workbench UI running.
- For each project optionally execute an action (supplied by a plugin in the workbench) that packages the build output into some external format (e.g. generating a WAR file from a WTP Dynamic Web project, generating JavaDoc from a Java project etc).
The last one above is a ‘nice-to-have’ since the invoking script may well be capable of finding the build output and packaging it using pre-defined rules.
Does anyone know of such a beast? Or is any effort underway to produce such a beast? Bugzilla doesn’t seem to have anything relevant which is a surprise. This seems like an awfully useful feature for the Eclipse platform to support…
on 19 Sep 2006 at 4:23 pm # Alex Blewitt
Maven is pretty much the default way to go for this kind of project, and you can automate it fairly easily and hook it up with cruisecontrol etc.
There’s also a bit of support for letting maven manage your .classpath entries in Eclipse; there’s a beta (or it may be released by now) of an Eclipse plugin that actually provides the .classpath entries direct from the maven pom.xml file (the dependencies list), so you only make the changes in one place.
Of course, it doesn’t satisfy the external builders, unless those external builders are kicked off by ant, or have an appropriate tool e.g. JibX compiler.
Frankly, the automated build in PDE is clumsy, and very specific to what it tries to do. Maven is very good, but has no real clue when it comes to OSGi bundles (though it’s coming). I suspect that a new maven-esque build tool will do all this and more, just as soon as I get around to writing it
on 19 Sep 2006 at 4:39 pm # Pascal Rapicault
The PDE Build team has had this in mind for the last 3 years. Unfortunatly it passed the stage of discussions and very simplistic prototype because of lack of interest and resources. One of the main idea I wanted to explore is the ability to use a script engine to interact with the underlying model of the workbench (for example to collect problems, sync, etc…).
On the other hand, one of the problem around this approach is scalability with badly behaving builders and other models being built too eagerly, but in the end everybody would benefit from them being fixed.
So to go back to your initial question, the closest thing I know about is buckminster which seems to have the ability to populate workspaces from of project set file on steroid, and apparently do build. Disclaimer: I have not tested it.
BTW thank you for showing love to PDE Build
on 19 Sep 2006 at 5:18 pm # aehso
Pascal,
The idea of a script engine to interact with the workbench model is along the lines of what I was thinking of - In fact, I have vague memories of someone at EclipseCon 2006 speaking about something similiar.
Buckminister looks interesting in so far as it certainly solves to problem of getting projects into a workspace (RMAPs do look like PDEBuild maps on steriods). Thanks for the pointer, I’ll have to investigate Buckminister-Headless ) a bit more, it may well be of use…
on 19 Sep 2006 at 5:33 pm # aehso
Alex,
I’m reasonably aware of Mavens support for building Eclipse projects but that support (for sync’ing .classpath files) is almost a different issue. The projects/builders I’m thinking about might have nothing to do with Java/JDT. They most likely store their metadata elsewhere and crucially they most certainly have to be run from within an initialized workbench.
(I’m touching on the religious debate of who owns the metadata - the build system (maven) or the components (eclipse projects) but lets not go there
Given the likelihood of more heterogeneous project builder sets in the future (especially with non-Java tools projects in Eclipse.org projects like CDT) this requirement will probably start appearing from all angles soon enough.
I also agree with Pascal’s other point - I’m sure the introduction of such a headless build framework might show up some problems with badly behaving builders but it’d be nice to be able to get that far, especially for those who don’t have use those offending builders!
on 19 Sep 2006 at 5:55 pm # oisin
BFB has blogged some stuff that came out of the Europa Workshop which mentions both Buckminster and Maven in passing: http://eclipse-projects.blogspot.com/2006/09/europa-build-workshop.html
Maybe we will get some interesting new ideas when he gets the chance to do detailed write-ups.
I have tested Buckminster, and it worked, but I’m not sure why exactly
Pascal, your script engine idea sounds very interesting, do you have a bugzilla for it by any chance?
on 20 Sep 2006 at 7:45 am # Philippe
ant4eclipse provides also a set of simple ant tasks to sync up a java project’s metadata with an ant script.
Here are some of your reqs:
>* Optionally import a set of projects into the target workspace
ant4eclipse has atsk cvsGetProjectSet that gets projects as defined in a Team Project Set from CVS.
>* Build all projects in the order determined by the declared project dependencies (in the project descriptor), running all builders as it would if the Workbench UI running.
ant4eclipse executeBuilders and getBuildOrder taks may help for that
It has a few more goodies . Check http://ant4eclipse.sourceforge.net/ant-for-eclipse-reference.html
Now I am not affiliated with that project.
on 22 Dec 2006 at 10:53 pm # Shankar
Hi,
Has anyone been able to get ant4eclipse to work in a “headless” mode? I have it installed in my eclipse subdirectory but unfortunately unable to make it work in “headless” mode.
The website for ant4eclipse doesn’t seem to have an example of running it in a “headless” mode.
Any pointers would be extremely useful.
Thanks
Shankar.
on 22 Feb 2007 at 9:46 pm # Markus
Hi,
In the past I had to set up PDE headless build for several projects. I created these build configurations by copying and pasting ANT scripts and I think that there are some common tasks which should be shared between the projects instead. Therefore I have started http://www.pluginbuilder.org.
One goal is to provide some convenience functionality, e.g. for creating map files (see “missing entries in the PDEBuild map file” above). There is already support for subversion repository access, release and nightly build configurations and test automation.
It would be nice if this would become a place where people share their scripts and Best Practices around PDE Build like site.xml maintenance, versioning recommendations or code instrumentation.
on 19 Sep 2007 at 4:12 pm # Tony
IBM Rational Application Developer (RAD) (built on top of Eclipse) has this feature “Running Ant in a headless workspace”
“You can use Ant to run a workbench with no interface and run specified Ant scripts.
In the root of the com.ibm.etools.j2ee.ant plugin is a sample batch file called runANT.bat. This .bat file will run a “headless” workbench (no user interface for the development environment) and run a specified ANT script file (example.xml). The ANT script file must be a fully qualified name.”
http://publib.boulder.ibm.com/infocenter/iadthelp/v7r0/index.jsp?topic=/com.ibm.etools.ant.tasks.doc/topics/tjantheadless.html
developerWorks article: Troubleshooting headless Ant builds with Rational Application Developer
http://www.ibm.com/developerworks/websphere/library/techarticles/0505_weisz/0505_weisz.html
Maybe we can ask IBM to donate this plugin to Eclipse.